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Rating  lo  a  proceae  in  which  we  relate  observed  perfor- 
aance  to  a  de&lgnated  concept  of  standard.  This  prooeee  Is 
usually  carried  out  to  determine  the  ''standard  tlHe".   Per<* 
foralng  the  job  In  the  standard  tlae  Is  called  "standard 
performance" • 

There  are  two  major  methods  of  rating, 
(1)  methods  u&lng  a  mathematical  formula, 
(3)  methods  using  judgement. 

The  mathematical  formula  methods  are  not  considered  as 
being  accurate  and  are  not  In  general  uee.   Industry  largely 
makes  use  of  tho^e  methods  involving  judgement  on  the  part 
of  the  rater. 

Time  study  englnecrF  in  Industry  today  have  been  on  the 
job  Various  lengths  of  time;  have  received  their  time  study 
education  from  various  sourceb,  end   utilize  different  con- 
oepts  of  "standard  performance".   It  if^  believed  that  their 
accuracy  In  rating  ic  atfected  by  these  and  other  factors 
such  as  area  of  employment,  slse  of  plant,  as  reflected  in 
number  of  employees,  and  size  of  city  In  which  plant  is  lo- 
cated.  The  objective  oi  this  thesis  wa^  to  determine  If  any 
or  all  of  thebe  factors  affect  the  accuracy  and  consistency 
of  the  ratings  made  by  a  typical  group  of  time  study  engineers 
from  the  Uld-West  region.   This  was  accompllehed  by  analyzing 
the  ratings  of  job  films  In  accordance  with  the  following 
breakdowns: 
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1.      Entire  group. 
3.     Area. 


Z. 


Kxperlence. 


4.  Initial  iource  of  time  study  training. 

5.  Concept  of   **Btandard  performance". 

6.  Size  of  plant. 

7.  Size  of  city  in  which  plant  is  located. 

9ith  the  engineers  asbetabled  in  a  room,  films  of  an 
operator  performing  a  synthetic  task  were  projected.   The 
job  was  a  simple  laboratory  operation,  representative  of 
light  factory  work.  The  films  were  divided  into  12  loops 
with  the  operator  working  at  a  different  pace  in  each  loop. 
Loops  were  projected  in  a  random  order  of  pace.  Engineers 
were  instructed  to  rate  by  cycle  nud  to  employ  any  method 
of  rating  that  taey  desired.  Method  of  performing  the  job 
was  stipulated  to  be  correct.   In  order  to  place  all  ratings 
on  a  mutually  comparable  basis  and  to  simplify  the  mathe-- 
aatical  problem.  Rating  Conversion  ^  caleis  were  used  by  each 
man  to  convert  his  raw  rating  to  that  figure  he  would  have 
assigned  had  he  assumed  130  to  be  the  maximum  numerical  value 
he  «ould  expect  of  the  typical  operator  in  his  plant. 

Conclusions. 

An  analysis  of  the  results  of  a  group  of  typical  time 
study  engineers  when  rating  job  films  ie&ds  to  the  following 
conclusions: 
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6.  Size  of  city.  Size  of  city  In  which  the  plant  1© 
located  apparently  has  little  or  no  bearing  on  the  r&tere* 
accuracy. 

9.   Cons  let  ency  follo'ved  by  the  same  direction  as 
accuracy  in  all  cases.   It  was,  however,  slightly  higher. 
This  waB  to  be  expected  fron  the  oianner  in  r^hich  the  data 
were  handled. 

This  analysis  suggests  that  when  rating  a  job  film  «e 
can  probably  expect  of  time  study  engineers,  in  terae  of  the 
percentage  of  personnel  within  limits  indicated,  the  follow- 
ing degrees  of  accuracy: 

k&H     level:   aore  than  one  third— 38^. 

+7V^  level:   almoBt  one  half 48^. 

+1<H  level:   alaoat  two  thirds 625^. 

*'20i»   level:   the  great  a^jorlty 86%. 
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AH  EVALUATION  OF  TIME  STUDY  PUTIROS  OF  A  SYKTHFTIC  TASK 

IHTRODUCTIOI 

"Motion  and  time  study  refers  to  the  analy^le  of  the 
work  and  tl«e  neceepary  to  do  a  certain  job*..   To  find  the 
tine  nece&aary  we  usually  resort  to  a  stop  watch  tlae  study. 
A  stop  watch  tlae  study  is  acoompllehed  by  observing  an 
operator  perfornlng  the  task  and  recording  the  tlae  obtained. 
However,  we  note  that  if  we  Bhould  time  another  operator  per- 
forming identioally  the  Base  job,  in  the  sane  atanner,  we  very 
probably  would  obtain  a  different  ti»e  of  accoapliahment. 
And  if  we  studied  a  tnird  operator  we  mlr:ht  get  a  third 
Value  of  tine.  The  question  now  arises,  "which  one  of  these 
workers  is  performing  the  task  in  the  correct  aaoimt  of  time"? 
The  answer  to  this  question  la   required  to  establish  a 
•standard  time*  for  the  job  in  question.   Performing  the 
job  in  the  standard  time  ia  called  "staniard  performance*. 

One  definition  of  standard  time  given  by  tfundel^  is, 
•the  time  necesbary  to  do  a  certain  job,  defined  ae  to 
method  and  conditions  surrounding  the  work,  by  a  operator 
physically  fit  for  the  job  as  the  typical  operator  who  could 
be  expected  on  the  job,  posfeeesing  sufficient  skill  to  do 


1  a.  E.  Mundel,  Systematic  Motion  and  Time  Study,  p.  3, 
Prentice-Hall.  Inc.,  Hew  York,  1947. 

2  Ibid.,  p.  147. 
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the  job  properly,  adapted  to  the  job,  and  working  at  a  pace 
100/ X30  of  the  maxlBuiB  he  could  fflain^^ain  for  the  working  day 
without  hiamful  physical  effects".   But  with  all  of  these 
conditions  understoo'^  and  defined,  it  is  still  necessary  to 
determine  what  relai ionahlp  the  performance  of  the  operator 
observed  bears  to  the  pace  defined  as  standard   The  urocess 
of  relating  observed  performance  to  a  designated  atanciard  is 
called  "rating",  and  further  discussion  will  be  confined  to 
this  subject. 

There  are  two  wajor  methods  of  rating, 

(1)  Methods  using  a  mathematical  formula, 

(3)  Methods  using  judgeaent. 

(a)  Multiple  factor  systeos, 

(b)  single  factor  nystene. 

Because  of  the  great  number  of  variables  affecting  the 
operators  tine  the  mathematical  formula  methods  are  not  con- 
sidered as  being  accurate,  are  not  in  general  use,  and  will 
not  be  oonaidered  here. 

An  example  of  the  judgement  method  i&  the  "effort* 
rating  plan.   This  plan  involves  two  steps, 

(1)  judge  the  difficulty  of  the  job  end  iaagiBe  a 
concept  of  reasonable  performance, 

(3)  compares  what  he  sees  to  this  concept. 
To  carry  out  this  method  the  time  study  man  should  be  faral- 
llar  'frith  the  visible  appearance  of  the  steps  in  the  range 
of  effort,  as  reflected  in  pace,  on  each  type  of  job  he  ob- 
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3 
serves.   In  a  plant  with  a  variety  of  jobs  this  is  not  always 

feasible. 

!?ow  judgement  must  involve  the  deteraination  of  the  effect 

of  the  operator's  skill,  aptitude,  and  effort  on  hie  perform- 

ance.   The  systen  advanced  by  Mundel  accoiBpli8he&  the  above 

and  entails  the  followiag  steps  on  the  part  of  the  rater, 

(1)  cooaparea  the  workers  pace  to  a  fixed  concept  of 
speed, 

(2)  adjusts  the  observed  value  for  difficulty  of  the  job. 
It  beooaes  obvious  that  a  standard  pace  nust  be  defined. 

The  standard  pace  'iiay  be  a  <tian  v^alklng  on  a  level  at  3  miles 
per  hour.   Al&o  it  could  be  a  person  dealing  a  deck  of  cards 
into  4  equal  stacks  in  one  half  of  a  minute.^  It  is  evident 
some  value  or  representation  of  pace  Bust  be  made  in  order 
that  judgement  of  pace  be  possible  and  effective.   A  physical 
embodiment  of  this  designated  u>ace  could  be  a  film  cf  an 
operator  walking  or  dealing  cards  at  the  respective  rates 
indicated  above.   Some  time  study  men  have  ^heir  own  concept 

of  standard  performance  and  some  utilize  a  t ilm  as  previously 

5 
mentioned.   When  using  the  Presgrave   a/stem  a  different  con- 
cept of  the  standard  pace  for  each  job  is  required. 


2   Ibid.,  p.  157. 

4  aunriel,  op.  cit.,  p.  157. 

5  Ralph  Presgrave,  Dynamics  of  Time  Study.  2nd  ed., 
p.  154,  .Mciiraw  Hill  Book  Co.,  Inc.,  Hew  York,  1945. 
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Rating  »a^   be  expxdtiaed  as  points  per  hour  or  as  a 
perce&t&ge  figure.   Percentage  numberR  are  Kore  widely  used 
and  will  be  employed  in  this  thesis.  There  are  various 
percentage  syeteae  ir.  use  but  the  folloiving  will  be  employed 
here:  the  aaxi«um  perlormanue  that  could  be  expected  of  the 
typical  worker,  for  tne  eu\.ire  working  day,  without  harmful 
physical  effects  will  be  150  and  standard  performance  will 
be  100/130  of  this  value. 

To  adjust  for  difficulty  of  performance  on  difierent 
jobs,  tables  are  prepared  with  certain  allowances  for  adjus- 
tment purposes. 

Time  study  engineers  in  industry  today  Lave  been  on  the 
job  Various  lengths  of  time;  have  received  their  time  study 
education  from  various  sources,  and  utilize  different  con- 
cepts of  "standard  performance".   It  is  believed  that  their 
accuracy  in  rating  is  affected  by  these  and  other  factors 
auch  as  area  of  employment ,  sixe  of  plant,  siie  of  city  in 
which  the  plant  is  located. 


6  Ralph  M.  Barnes,  Motion  ruid  Time  Study.  3rd  ed., 
p.  353,  John  Wiley  *  Sons,  Inc.,  Hew  York,  1949. 
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OBJECT 

The  object  of  this   thesis   is  to  determine  if  any  or  all 
of  these  aforementioned  factors  affect  the  accuracy  and  con- 
sietency  of   ratings  ai&de  by  typical   group  of  tiae  btudy 
engineers  froa  the  i/id*?:e6t  region.      This   object   »ill  be 
invet^tigated  by  sLiialyzing  the  ratings  of    job  filne   in 
aocord&nce  with  the  following  breakdovne: 

1.  Kntire  group. 

2.  Area. 

a.  Northern  Mid- West,  less  Michigan. 

b.  Central  J^id-Weet. 

c.  Southern  Mld-^eet. 

d.  yiohigan. 
2.   £xperidnce. 

a.  0>6  months. 

b.  6  taonths  -  3  years, 

c.  3-4  years. 

d.  Over  4  years. 

4.  Initial  source  of  tiae  study  training. 

a.  College. 

b .  Ooapany . 

5.  Concept  of   "standard  performaiiCe". 

a.  Rater  u&ea  own  coucept. 

b.  Rater  uses   a  fila  or  8ome  other  e!rtbodinent. 

6.  Size  of  plar^t,    (number  of   employees). 

a.  C-lOi  men. 

b.  101-1000  men. 
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c.  Over  1000  men. 
7.      Size  ot  city   In  irblch  pXant   la  located. 

a.  Under  5000. 

b.  5000-10000. 

c.  10000-31000. 

d.  35000-50000. 
a.  50000-100000. 
f.  Over  100000. 
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^0C5DURK-PaRT  a 

In  order  to  carry  out  the  objectives  of  thia  theelB  it 
was  necessary  to  hairs  a  t/pio&l  group  of  tine  study  •nfciaeere 
rate  a  task  or  a  nuaber  of  tasks.   To  obtain  this  data  In-> 
dustrial  Engineers  from  various  firms  located  in  Indiana, 
Illinois,  Michigan,  Ohio,  and  Wisconsin  were  invited  to  attend 
a  "work  session*  at  Purdue  University  on  the  15th  of  »^arch, 
1950. 

Data  could  have  been  obtained  by  having  the  raters  view 
either  a  fil»  of  an  operator  performing  a  t?»8k  or  view  an 
actual  operator.   For  various  reasons  a  t 11m  was  cnosen  rather 
than  en  operator.   Some  of  these  reesor.s  are: 

1.  A  film  was  already  available  thus  obviating  the 
necftfisity  of  training  an  opRrotor. 

2.  A  film  is  more  convenient  as  it  can  be  viewed  by  a 
much  larger  group  than  could  an  operator  working  at  a  task. 

3.  Meti-od  and  Bklll  can  be  maintained  more  consiBtently 
and  accurately  by  means  of  a  film  since  many  operators  are 

distracted  ^heR   performing  in  the  presence  of  aa  audience. 

7 

4.  It  was  shown  by  Margolin  that  time  study  engineers 

rat^e  more  accurately  and  consistently  from  a  job  film  than 
from  an  operator. 

The  film  selected  consisted  of  an  operator  performing  a 
simple  task,  a  job  description  of  which  is  ap^^ended.   The 


7  Louis  !i4argoiin,  "A  Ooaparison  of  Two  Methods  of  Pre- 
sentation For  Time  f^tudy  Ratings*,  Unpubliened  Uasters   Thesis, 
Purdue  University,  1948. 
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film   *aB  prepared  in  i9^ir9  unaer  the   supervision  of  Lehrer. 
It  '>«as  diviciod  into  36  parts  in  which  the  operator  performed 
at  a  different  pace  in  each  part.   The  f ila  had  been  carefully 
edited  to  ellalnate  any  fumbles,  false  wovements  or  ?ny  other 
irregularitlaa.   OonsiderationB  involved  in  the  selection  of 
this  filai  ware: 

1.  It  involvsa  &Uarp  clear  actions  and  elements. 

2.  Includes  sotlone  of  the  hands  and  arns  that  are 
typical  of  light  factory  work. 

Twelve  loops  were  prepared  froa  the  fila.   They  were 
carefully  checked  to  insure  that  the  ratio  between  the  elea-* 
entB  of  a  given  loop  in  each  cycle  were  the  same.  Loops  were 
arranged  in  a  randoa  order  of  pace  for  projection  purposes. 

It  being  realized  that  different  time  study  men  utilise 
diflerent  nuaerioal  aaximua  percentage  values  as  expectable 
of  the  typical  worker,  rating  convention  scales  were  prepared 
for  their  use.   In  this  manner  they  could  convert  their  raw 
rating  to  the  one  in  which  130  represents  the  typical  u&ximua. 

Two  IBM   sark  sensin^^  cards  were  prepared  for  issuance 
to  each  saan.   These  cards  were  punched  vlth  identifying  date 
for  each  m&n.   It  was  arranged  that  each  man  would  then  enter 
his  "converted  rating*  directly  on  the  IBfc  cards. 


8  R.  K.  Lehrer,  '*t)evelopaent  and  Evs-luation  of  a  Pace 
Scale  for  Time  Study  Rating**,  Unpublished  Doctors  Thesis, 
Purdue  University,  1949. 
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proci:dure-part  b 

Th«  engineers  assembled  in  the  lecture  rooa  of  the 
Electrical  Snglneering  building  and  after  a  greeting  by  Dr. 
M.  S.  aunael.  Professor  of  Industrial  Engincsrizig,  General 
Engineering  Department,  Purdue  University,  a  questionnaire 
«aB  iianded  out  to  each  oan.   This  questionnaire  was  for  the 
purpose  of  collecting  inforoiation  for  the  thesis  of  Borrus.^ 
It  further  served  the  purpose  of  procuring  the  necessary 
infomiation  such  as  area,  experience,  etc.,  of  each  aan  for 
this  thesis. 

On  coapletion  of  the  questionnaire  the  IBM  nark  sensing 
cardB,  special  pencils,  rating  conversion  Bcalec,  were  issued 
to  each  man.   Instructions  for  their  use  was  given. 

Engineers  were  requested  to  rate  by  cycle,  using  their 
own  system  of  rating.   Method  of  the  job  was  stipulated  to  be 
correct.   Tney  were  further  reminded  to  use  the  rating  con- 
version scales  in  order  that  all  ratings  would  be  on  a 
nutually  comparable  basis. 

A  loop  was  projected  for  a  short  time  to  familiarise 
each  man  with  the  method  of  the  job.   On  completion  of  this 
showing  the  session  began.   Loops  were  projected  at  the  ^ame 
speed  at  which  they  were  made.   About  3  minutes  were  allov;ed 
for  projection  and  about  1  minute  ior  recording  of  ratings. 
This  procedure  was  continued  until  all  12  paces  were  shown. 


9  Bernard  S.  Borrus,  "Tne  Present  State  of  Time  Study ♦*, 
Unpublished  Masters   thesis,  Purdue  University,  1950. 
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PROCEDURE- Part  0 

After  all  13  paces  had  been  rated  the  IBK  cards  were 
delivered  to  tho  TBti   Coniputing  Laboratory  where  all  necessary 
data  were  punched  on  the  cards.  The  IBH  Laboratory  supplied 
the  following  inforaation  lor  each  pace  and  for  each  categ;* 
ory  of  breaicdown: 

1.  number  of  raters. 

3.  Totalis  of  their  scores. 

3.  All  ratings  made. 

4.  Suaber  ol  ratera  with  a  specific  score. 

With  this  inforiA&tion,  corrected  values  for  each  pace  «m* 
determiiied.   Uatheaiatical  procedure  for  determining  this  in   con- 
taiiied  in  Appendix  B.   ^ith  the  corrected  value  of  each  pace 
obtain edy  and  the  information  fro»  IBM,  analyzation  fur 
accuracy  was  made  by  noting  the  number  and  percentage  of 
raters  within  ±pi,   ±7^^,  ±10^,  i.20^  of  the  corrected  values 
for  each  pace  and  for  each  category  of  breakdown. 

To  analyze  for  consistency,  the  same  procedure  was  enploy- 
ed  except  tne  average  of  the  raters'  values  was  used  a^-:  the 
•easure  about  which  the  variation  was  appraised. 

To  arrive  at  conclueions  the  data  were  invettigateri  by 
Inspection  and  by  the  "analyBie-of -variance*  techniaue.   The 
analysia-of-variancft  technique  waB  employee  at  the  ±5^   and 
±10li   level  of  accuracy  in  all  ca^es  and  at  the  4.20^^  in  a  fer. 
Appendix  3  has  a  Bhort  dlBcussion  on  the  anal ysis-of- variance 
technique. 
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Raters*  averages  were  plotted  against  the  corrected 
valueB  for  each  category  of  breakdown.   A  line,  determined 
Dy  tne  method  of  *lea.st  scares",  was  drawn  tnrough  these 
points.   Thib  line  indicated  the  points  at  which  raters 
tenaed  to  depart  from   the  ±5is   and  ±XO^   levels  of  eccuxacy. 
Appendix  B  contains  a  short  diucuesion  on  the  ♦•ieaet  squaree" 
method  of  fitting  &  curve  through  a  ^et  of  points. 
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COKCLUSIOHS 

An  analysis  of  the  result 6  of  a  group  of  typical  tiai* 
study  engineers,  when  rating  Job  filas  of  various  pecee, 
leads  to  the  following  conclusiozie: 

1.  For  all  oategorieB,  a  plot  of  the  average  of  the 
ratings  against  the  corrected  ratings  indicates  a  treed  of 
tae  tiae  study  erji^ineers  to  rate  too  low  gt  the  higher  values 
and  too  high  at  the  lower  values. 

2.  The  line  beet  fitted,  by  mathenatical  seans,  to  all 
points  for  the  entire  group,  suggest  the  trerd  of  this  group, 
on  the  average,  to  be  outside  the  +5^  level  at  values  below 
about  93;  the  410^  level  at  values  below  about  75;  the  -6^ 
level  at  values  greater  than  165;  the  -104  level  at  values 
in  excess  of  320. 

3.  Area.   The  Michigan  oree  had  a  comparatively  low 
degree  of  acoursoy,  the  anal /sis-of-var lance  teet  £howed  it 
to  be  aignificant,  but  Bince  their  anount  of  experience  n^nd 
familiarity  with  the  method  of  rnting  a  job  film  was  not 
deterifiiuea,  and  the  sample  was  very  small,  their  score  is 
disregarded.   Discounting  the  Jtichigan  category,  raters  froa 
various  areas  rated  easentially  the  aase.   Therefore,  the 
area  in  whicn  the  plant  id  located  appears  to  have  little 

or  no  relationship  to  accuracy. 

4.  Experience.   Groups  with  6  months  or  more  experience 
rated  about  the  saoie  but  «itU  greater  accuracy  tLan  those 
with  leas  experience.   Those  in  the  3-4  yeara  category  were 
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Bligiitly  higher  tiian  the  r«8t.   Tlie  enalyalB-of-varipnce  test 
failed  to  indicate  that  the  low  score  of  the  0-S  raor.the  pro  up 
was  significant,  but  in  view  of  the  irregularity  cf  their 
accuracy,  a«  indicated  by  Inspection  of  Tsble  7a,  it  is 
eugc^ested  that  in  general  at  lea^t  6  aonthB  experience  is 
reouired  to  make  the  Qualified  time  study  engineer  but  that 
accur;»icy  then  reeiaine  about  the  esiae  with  increp.sed  experience. 

^*      ^^ducation.   The  accuracy  of  the  raters  at  the  45- 
level  ie  practically  equal.   The  1^  difference  ir  their 
accuracy  is  not  significant.  However,  at  the  £10^  level 
taoJ^e  raters  receiving  their  initial  tine  study  training 
froB  a  company  Bhow  a  significantly,  by  )*naly  els- of -variance 
test,  greater  accuracy.   This  sug^s^ests,  at  leaet  for  this 
group,  at  the  higher  level  of  accuracy,  initial  &ource  of 
tl«e  &tudy  training  doeis  not  affect  accuracy.   At  a  les8 
rigid  degree  of  accuracy,  tiae  etudy  men  receiving  ir.it ial 
time  etudy  training  from  the  coapany  are  more  accurate  than 
those  receivinpj  it  froa  a  school.   ThiB  cay  not  be  true  of 
those  receiving  modern  trainiiig  from  either  echool  or 
company. 

6.   Oonoept  cf  Standard  Performance.   Rat  era  using  a  film 
or  some  other  embodiment  of  standard  performance  as  comraon 
practice  were  superior  to  thor^e  using  their  own  concept 
despite  tne  fact  that  no  film  aid  was  used  in  this  oha&e.  The 
difference  in  accuracy  wefc  not  tignif leant  at  the  ±5i.   level 
of  accuracy  but  was  definitely  Bignificnnt  at  the  ±10^   level. 
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It  is  therefore  8uj.-pe8t*?d  that  raters  should  uee  a  film  or 
sotte  other  eMbodi««nt  of  "standard  pert orraanoe* .  and  thle 
reinforces  the  last  alpitement   of  conclui^ion  5. 

7.  Nuaber  of  RMployeetJ.   By  in&pection  the  ret  ere  in 
pia&ts  eapXoyirig  less  than  101  mex)  rated  a  little  lower  than 
those  in  larger  plcnte.   However,  this  difference  ie  not 
aignif icaiit.   Therefore  the  sire  of  plant  a^  reflected  in 
number  of  plant  employeeB  apparently  has  no  effect  on  raters 
accuracy. 

8.  Size  of  city  in  which  plant  ie   located*   Inepection 
reveals  a  «ide  difference  in  accuracy  between  s  city  of  less 
th<in  5,000  people  and  a  city  of  5,000-10,000  people;  the 
anal yiJi8-of -Variance  teats  reveals  this  ditfererice  to  be  in- 
significant at  the  +5^  level  of  soouracy  an^  barely  significant 
at  the  £10^  level.   It  ia  therefore  suggested  that  sife  of  city, 
in  which  the  plant  is  located,  does  not  af 1 ect  raters  accurecy 
significantly. 

9.  Consistency  followed  by  the  sane  direction  as  accuracy 
in  all  casei>.   It  wa^,  however,  slightly  higher.   This  was  to 
be  expected  from  tne  i^iethod  in  which  tne  data  were  handled. 

10.  This  analysifc  sugget^ts  that  when  rating  a  job  fil« 
we  can  probably  expect  time  study  engineers  in  terms  of  the 
percentage  of  pereoni.ex  tr^ithin  limits  indicatea,  to  perform 
with  the  following  degrees  of  accuracy: 

^5i     level:   more  thttn  one  third — —28^, 
jj^i   level:   almost  one  half 4P4. 
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^10^  level:      almost  i%o  thirds 62^. 

irZQTji,  level:      the  great  majority 86^. 
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Table  la 

Breakdown:   Tabulation  of  Nuaerioal  Results  of  all  Categorii 
Computed  for  Accuracy 


±51^ 

tm 

±10f, 

±20i 

Ho. 

* 

Kb. 

i 

Wo. 

i 

iro. 

€ 

Overall. 

37 

38 

35 

48 

45 

68 

63 

86 

Northern  Mid-'^est, 
Less  Michigan. 

7 

37 

11 

53 

13 

61 

17 

85 

Central  Mid-^est. 

13 

38 

17 

55 

20 

65 

27 

89 

Southern  Mid-Weot. 

5 

43 

7 

59 

9 

73 

11 

88 

Michigan. 
Experience, 
0-6  oos. 

3 

3 

31 
33 

3 
3 

36 
47 

4 

4 

45 

58 

7 

5 

77 

78 

6  mob.-3  yrs. 

7 

43 

9 

53 

10 

64 

14 

86 

2-4  jrrs. 

7 

39 

10 

54 

13 

66 

18 

93 

Over  4  yrs. 

Education. 

College. 

11 

8 

39 
36 

16 
11 

53 
49 

18 
13 

63 
59 

36 
18 

89 
81 

OoBipany 

16 

38 

34 

56 

38 

66 

39 

87 

Uae  o»n  Concept  of  20 
Standard  Performance 

37 

38 

51 

34 

63 

49 

87 

Uses  f  iiai  or  other   7 
embodiment  of 
Standard  Performance 

44 

9 

61 

11 

75 

15 

98 
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Table  lb 


Breakdown:   Tabulation  of  Fumericel  Results  of  all  Categories 
Computed  for  Qonfcistenoy 


+51. 

in^ 

tio* 

420i 

Ko. 

i 

So. 

i 

!Jo. 

i 

Ho. 

4 

Overall . 

39 

40 

41 

57 

50 

69 

62 

86 

Horthern  Llid-^est, 
Lees  Michigan. 

8 

39 

11 

53 

IS 

66 

17 

86 

Central  Mid-feet. 

13 

42 

18 

56 

22 

73 

27 

88 

Soutnem  Uld-Wefct. 

6 

51 

8 

63 

9 

74 

11 

88 

Michigan . 
Experience, 
0-6  oos. 

3 

3 

3X 

35 

3 

3 

37 
51 

4 
4 

48 
65 

6 
5 

71 
82 

6  BOS. -2  yra. 

6 

4N> 

9 

57 

11 

66 

14 

87 

3-4  yra. 

a 

ao 

13 

61 

13 

70 

17 

91 

Over  4  yrs. 

Education. 

College. 

8 

44 

38 

XT 
13 

58 
54 

30 
14 

67 

64 

26 
18 

90 

82 

Ooapany. 

10 

4S 

3T 

60 

32 

71 

40 

88 

Use  own  Concept  of  30 
Standard  Perfcrraaiice 

38 

29 

52 

37 

68 

47 

87 

Uses  film  or  other   8 
•mbodi:Bent  of 
Standard  Perforaance. 

56 

11 

72 

13 

64 

15 

98 
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BreaJcdown:   S&tlre  Oroup. 
number  of  Rat ere:  73 
Evaluation  for  acouraoy. 
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Corrected  Sequence 
Ratings.      Shown 


155 

12 

143 

4 

137 

11 

132 

8 

122 

1 

118 

5 

105 

10 

98 

2 

95 

6 

92 

3 

81 

9 

79 

7 

Average: 

± 

.5* 

±7i1t 

410^ 

+20^ 

Mo. 

i 

Vo. 

a 

Wo. 

* 

Vo. 

i 

29 

47 

46 

64 

48 

67 

66 

93 

23 

32 

32 

44 

47 

65 

67 

93 

38 

53 

46 

64 

57 

79 

66 

92 

34 

47 

44 

61 

50 

69 

63 

87 

31 

43 

45 

65 

56 

78 

67 

93 

30 

42 

45 

65 

52 

73 

65 

90 

35 

34 

32 

44 

43 

58 

63 

87 

31 

43 

40 

55 

49 

68 

64 

89 

36 

36 

32 

44 

42 

58 

57 

79 

24 

33 

28 

39 

35 

49 

60 

83 

17 

24 

31 

29 

26 

39 

53 

74 

18 

35 

26 

36 

32 

44 

54 

75 

27 
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Breakdown:     Entire  Group 
Number  of  Haters:      73 
Svaluation  for  cousiBtency. 
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Average 
Ratings 

Sequence 
Snown 

±5^ 
Ko. 

i 

No. 

No. 

* 

4204 
Ho.  i 

149 

13 

36 

60 

44 

61 

53 

73 

64 

89 

1S5 

4 

35 

49 

49 

66 

63 

66 

68 

95 

135 

U 

96 

60 

48 

67 

59 

63 

66 

9S 

130 

8 

30 

43 

45 

63 

48 

67 

63 

68 

121 

I 

33 

46 

46 

67 

56 

61 

67 

93 

115 

5 

33 

46 

43 

60 

64 

75 

66 

93 

110 

10 

36 

36 

43 

56 

51 

71 

61 

85 

103 

a 

37 

37 

43 

60 

46 

67 

64 

89 

101 

6 

88 

39 

37 

51 

44 

61 

58 

81 

98 

3 

m 

31 

41 

67 

49 

66 

61 

tm 

86 

9 

19 

36 

33 

31 

3X 

50 

54 

76 

84 

7 

30 

36 

36 

39 

34 

47 

50 

79 

Average: 

36 
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50 

694 
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5* 
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d 
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e# 

6£ 

as 

ax 
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ea 

^8 

V8 

8^ 

08 

5|k 
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vs 

s 

SOX 

IB 
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13 

^^ 

xa 

V5 

G£ 

88 

8 

xox 

68 

r^ 

82 

V3 

JL^ 

X5 

ss 

S 
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IS 

IS 
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8V 

08 
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Table  3a 

Breakdown:      Area-Morthern  Mid-Weet,    less  Mlchlfran. 
Ifumber  of  Hat  era:      20 
Evaluation  for  accuracy. 


Corrected 
Ratings 

Sequence 
Shown 

So.  i 

So.  < 

±1X4 

So.  ^ 

So.  $ 

152 

13 

5 

35 

12 

60 

13 

60 

19 

95 

140 

4 

8 

40 

12 

60 

13 

65 

18 

90 

134 

11 

9 

45 

13 

60 

17 

85 

18 

90 

139 

8 

9 

45 

13 

80 

14 

70 

17 

85 

119 

X 

15 

80 

17 

85 

18 

90 

20 

100 

115 

5 

10 

50 

14 

70 

15 

75 

18 

90 

102 

10 

4 

30 

5 

35 

7 

35 

17 

85 

96 

3 

7 

35 

13 

65 

13 

65 

19 

95 

92 

6 

8 

40 

9 

45 

9 

45 

16 

80 

90 

3 

6 

30 

10 

50 

11 

55 

17 

85 

79 

8 

4 

30 

6 

30 

9 

45 

13 

65 

77 

7 

3 

15 

6 

30 

10 

50 

14 

70 

Average: 
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table  2b 

Breakdown:      Area-Korthern  iCid-Webt,    ies**  Michigan. 
Biusber  of   Rat  ere:      30 
Svaluation  for  coneistency. 
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Average 
Ratings. 

Siequence 
bhown 

Ko. 

& 

i 

No. 

It 

+  10^ 
Ho.  % 

No.   1L 

147 

13 

6 

30 

8 

40 

11 

55 

19 

95 

132 

4 

14 

70 

16 

80 

18 

90 

19 

95 

134 

11 

8 

40 

11 

56 

13 

65 

18 

90 

127 

8 

8 

40 

13 

60 

14 

70 

17 

85 

130 

1 

15 

75 

16 

80 

18 

90 

20 
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113 

6 

11 

55 

13 

65 

15 

75 

18 

90 

106 

10 

4 

30 

8 

40 

10 

50 

17 

85 
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3 

7 

35 

11 

55 

16 

80 

19 

95 
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6 

5 

35 

7 

35 

14 

70 

15 

75 

94 

3 

9 

45 
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50 
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90 

80 

9 

4 

20 

7 
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9 

45 

14 

70 
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e 
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V 
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06 
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as 

V 

oc 
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e? 
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Table  4a 

Breakdown:      Area-Central  tfid-TTest. 
Huaber  of  Ratsrs:      31 
ISvaluatlon  for   nccuraoy. 


Corrected 

Ratings 

Sequence 

So.  "^^ 

Ko.   4 

tioi 

?To.  'i 

f20i 
Ho.  t 

150 

13 

19 

61 

30 

65 

33 

74 

30 

97 

139 

4 

13 

39 

17 

65 

31 

68 

30 

97 

133 

U 

IT 

58 

37 

87 

88 

90 

30 

97 

138 

8 

13 

43 

18 

58 

27 

87 

39 

94 

118 

1 

13 

43 

33 

74 

37 

87 

39 

94 

114 

5 

15 

48 

30 

85 

35 

81 

31 

100 

102 

10 

8 

36 

15 

48 

16 

53 

35 

81 

95 

a 

15 

48 

19 

61 

21 

68 

35 

81 

93 

6 

9 

39 

IS 

43 

14 

45 

26 

84 

90 

3 

8 

38 

14 

45 

15 

48 

36 

84 

79 

9 

ft 

16 

9 

39 

11 

38 

88 

74 

76 

7 

6 

19 

9 

39 

14 

45 

35 

81 

kteTB-gei 


18       S8i     17       98$     m       6991     27       89^ 
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te     01 
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SI 

IS 
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#t: 

ODI 

1"^ 

r!S 

i.«J 

c;a 

LC 

18 

as 

83 

h&^ 


•  <i ":•  i  "vt* \ ' 
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Table  4b 

BrAakdom:      Area^Oentral  Mid^Veet. 
Number  of  Rati ars:     31 
STaiuation  for  const etency. 


Average 
Ratings 

n  equenoe 
Shown 

4 

lo. 

5* 

4 

lo. 

7^^ 

110^ 
Mo.  ^> 

420^ 
No.   ^ 

143 

12 

13 

42 

18 

58 

27 

87 

29 

94 

131 

4 

14 

46 

22 

71 

39 

94 

31 

100 

139 

11 

18 

SB 

24 

78 

37 

87 

31 

100 

126 

6 

12 

39 

30 

65 

36 

84 

39 

94 

119 

1 

13 

42 

33 

71 

27 

87 

29 

94 

111 

5 

13 

42 

30 

65 

33 

74 

30 

97 

107 

10 

12 

39 

17 

55 

24 

78 

38 

91 

101 

2 

13 

42 

15 

48 

22 

71 

34 

78 

100 

6 

16 

52 

18 

58 

19 

61 

27 

a7 

95 

3 

13 

42 

14 

45 

19 

61 

26 

84 

84 

9 

10 

32 

14 

45 

14 

45 

33 

71 

82 

7 

11 

35 

12 

39 

13 

43 

21 

68 

Average: 

13 

49^ 

18 

58^;«' 

22 

73^ 

27 

8» 

a^  9AQ&T 


.o.\ 
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IT 
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U 

te 
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51 

i^ 
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«V 
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vx 

es 

SX 

ST 
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XV 

SK 
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ai 

^■f> 

M 

TB 
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Xd 

ei 

8c 

8X 
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31 

*■« 

ds 

xe 

ex 

a* 

>X 

f 

5X 

It 

SR 

^^* 

*x 
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^X 

■ 

OX 

88 

xs 

s,^ 

sx 

GS 

fciX 

d6 

Xi 

Table  5a 

Breakdown:   Area— —Southern  aid-?eat. 
Huaber  of  Raters:   12 
Kvaluation  for  accuracy. 


Corrected 
Ratings 

^e-^uanoe 
51iown. 

JIo.  % 

if* 

Ko. 

VIO^ 
No.   ^ 

421 
Ko. 

0$ 

165 

X3 

3 

35 

9 

75 

10 

82 

12 

100 

152 

4 

4 

33 

7 

58 

7 

58 

12 

100 

146 

11 

10 

B3 

10 

8S 

10 

82 

11 

92 

140 

8 

8 

67 

9 

75 

10 

83 

11 

92 

ISO 

I 

3 

25 

6 

42 

9 

75 

12 

100 

126 

5 

6 

SO 

6 

50 

9 

75 

11 

92 

112 

10 

5 

42 

9 

75 

9 

75 

10 

83 

105 

3 

2 

17 

6 

50 

9 

75 

10 

8S 

101 

6 

5 

42 

6 

67 

8 

67 

10 

83 

99 

3 

6 

50 

7 

58 

9 

75 

11 

92 

87 

9 

3 

17 

Z 

25 

6 

50 

9 

75 

84 

7 

6 

50 

6 

50 

7 

58 

e 

67 

Average: 

» 

42^ 

7 

59^ 

9 

72f 

11 

88f 

?i 

•^.f 

9 

m± 

•1 

^i'?t 

^ 

^t 

•  - 

.  0 : ' 

.e>|^. 

^, 

,Oi{ 
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• , 
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31 

6f 

e 
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S 
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Bi 

? 

S<,. 
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-■ 
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Oi 
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8 

ti 

8^ 

9 
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c« 

QU 
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XI 

dt 

0 

Od 

d 

ijc 

B 

m 

IH 

«t 
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& 
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^ 

% 
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n 
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K  aox 

s 
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Table  5b 

Breaicdown:      Area Southern  Mid-Weet. 

Hujaber  of   Raters:      12. 
Svaiuation  for  consistency. 


e  uence 
iiown 

^5% 

Ro.  f 

+105 

No.  ;.. 

Ho. 

159 

IS 

10 

83 

10 

83 

11 

92 

11 

98 

144 

4 

e 

SO 

8 

67 

11 

98 

11 

93 

145 

11 

10 

83 

10 

83 

11 

93 

11 

93 

141 

8 

8 

67 

9 

75 

10 

83 

11 

93 

125 

I 

6 

60 

6 

50 

7 

56 

11 

93 

125 

5 

6 

50 

6 

50 

9 

76 

10 

83 

117 

10 

5 

43 

7 

88 

8 

67 

11 

83 

106 

z 

8 

17 

8 

50 

7 

58 

10 

63 

102 

6 

6 

48 

8 

87 

8 

87 

10 

83 

106 

3 

6 

50 

9 

75 

9 

75 

11 

92 

96 

9 

5 

48 

6 

50 

8 

67 

9 

75 

92 

? 

4 

33 

8 

49 

7 

58 

10 

83 

ATera^e: 

6 

Slit 

8 

88f 

8 

74* 

11 

88" 
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iX 

se 
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S8 

01 

SI 
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m 
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?0 

n 
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:# 

;.  K   ■' 
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rx 

-'' 

'Ji 

m 

01 

c.^1 

«e 
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01 

d^ 

e 

Ta 

8 

8 
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m 

03 

8 

oa 

a 

J. 
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d8 
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a 
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a 

& 

$€ 

e 

da 

T 
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d 

OX 

m 

01 

Rf? 

r 

Q<2 

8 

tr 

er 

€ 
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ij 

^ 

« 

u^ 

■£i 

B 

SOI 

av 

e 

e 

Od 

« 

« 

601 

av 

e 

73 

a 
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e 
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58 

ox 
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Table  6a 


Breaicdown:      Area-' — Michigan. 
:^U3iber  of  Raiere:      9 
Evaluation  for  accuracy. 


Corrected 
Ratings 

^?8C!uence 
Siiown 

!T0,  $ 

sro. 

At 

TO 

410^ 

Ko.  -^ 

±20^ 

lo.  ii 

165 

xa 

1 

il 

3 

33 

4 

44 

9 

100 

152 

4 

1 

11 

2 

33 

3 

33 

8 

88 

145 

11 

3 

22 

4 

44 

5 

55 

8 

88 

140 

8 

I 

IX 

3 

33 

5 

33 

7 

77 

130 

1 

1 

XI 

a 

33 

3 

23 

5  ■ 

55 

125 

5 

2 

22 

4 

44 

4 

44 

6 

66 

ill 

10 

3 

53 

6 

66 

6 

66 

6 

66 

104 

3 

1 

11 

3 

35 

4 

44 

7 

77 

101 

6 

4 

44 

4 

44 

5 

55 

7 

77 

98 

S 

4 

44 

4 

44 

4 

44 

7 

77 

86 

9 

S 

23 

3 

22 

6 

66 

7 

77 

84 

7 

1 

11 

S 

83 

4 

44 

6 

66 

Average: 
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« 
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s 

Table  6b 


Breakdown:      Area k^lcliigan. 

Humber  of  Raters:   9. 
Evaluation  for  con«lBtency. 
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Average 
Ratinga 

Sequence 
Shown 

Ko 

No. 

±10% 

No.  % 

4301t 

Ko.  i 

158 

la 

44 

5 

55 

6 

66 

8 

88 

145 

33 

3 

33 

6 

66 

7 

77 

144 

XI 

33 

4 

44 

5 

55 

7 

77 

134 

33 

4 

44 

4 

44 

7 

77 

133 

33 

3 

33 

4 

44 

6 

66 

133 

44 

4 

44 

4 

44 

6 

66 

116 

10 

33 

4 

44 

5 

55 

6 

66 

107 

a 

1^1 

3 

33 

3 

33 

7 

77 

107 

6 

33 

3 

33 

4 

44 

6 

66 

111 

8 

33 

3 

33 

3 

33 

6 

66 

9S 

9 

33 

3 

33 

5 

55 

6 

66 

93 

7 

33 

3 

33 

3 

33 

5 

55 

Average: 

311t 

3 

37* 

4 

481^ 

6 

7V 

86 


rfa  ^X-aat 

.9      :Q^e^aJl  to  'xedsifa 
***i^  |6^  ^^  «^i         «»ii»4ijtef5       ©:^ai«»vA 
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T?  T  era  a  »#  *  >;?;  r^;  xx  m»x 
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3   88   8    SS    S   88    8        T        (^ 
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Table  7a 

Breakdown:   Hlxperienoe- — 0-6  Months. 
Number  of  Raters:   6 
SvaXuatlon  for  accuracy . 
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Corrected 
Ratings 

Bequence 
Shoim 

Ho 

HO.  It 

twt 

No.  % 

Ko.  i 

152 

13 

4 

67 

4 

67 

4 

67 

6 

100 

140 

4 

3 

50 

4 

67 

4 

67 

6 

100 

134 

11 

3 

50 

4 

67 

5 

83 

6 

100 

139 

8 

2 

33 

3 

50 

5 

83 

5 

83 

130 

I 

4 

67 

4 

67 

5 

83 

6 

100 

116 

6 

3 

50 

4 

67 

4 

67 

5 

83 

103 

10 

1 

17 

3 

33 

3 

50 

5 

83 

96 

8 

3 

50 

3 

50 

4 

67 

5 

83 

93 
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0 

1 

17 

3 

33 

4 

67 

91 

8 

0 

0 

3 

35 

3 

50 

3 

50 

80 

9 

1 

17 

1 

17 

1 

17 

3 

50 

77 

7 
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0 

a 

33 

a 

33 

3 

33 

Average: 
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fB 

» 
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■> 

oa 
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« 

«0 

tB 
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BZ 

S 
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X 
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8 

89 
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5 
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Table  Tb 

Breakdown:      Experience- — 0-6  Months. 
Kumber  ol   Raters:      6. 
HIvaXuatlon  for  ooneietenoy. 


Average 
Hat  Inge 

Sequence 
Shown 

Ho. 

It 

Ho.   ^^ 

±104 
Ho.  % 

Ho.  i 

145 

13 

3 

33 

4 

67 

6 

100 

6 

100 

133 

4 

3 

33 

4 

67 

6 

100 

6 

100 

131 

11 

1 

17 

4 

67 

5 

83 

6 

100; 

139 

8 

3 

33 

3 

50 

5 

83 

5 

63 

133 

I 

3 

50 

4 

67 

5 

83 

6 

100 

110 

6 

3 

33 

3 

33 

3 

33 

4 

67 

108 

10 

3 

50 

3 

50 

3 

50 

5 

83 

103 

8 

4 

67 

4 

67 

4 

67 

5 

83 

96 

6 

1 

17 

3 

33 

4 

67 

4 

67 

95 

3 

3 

50 

3 

50 

3 

50 

4 

67 

84 

9 

0 

0 

3 

33 

3 
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4 

67 

83 

T 
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33 

3 

33 
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67 
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Table  8a 

Breakdown:   Experience-— >6  Months  to  3  Year^. 
Number  of  Raters:   16 
Evaluation  for  acouracy. 


Corrected 
Ratings 

3e  !uenoe 
Shown 

Ho 

i9* 

Wo.     i 

No.  % 

£20^ 

iro.  i 

166 

3J 

80 

9 

88 

18 

75 

16 

100 

144 

4 

ST 

8 

56 

11 

68 

14 

88 

158 

U 

81 

8 

66 

11 

69 

14 

88 

133 

• 

60 

10 

88 

11 

69 

14 

88 

123 

1 

80 

8 

58 

10 

63 

15 

94 

119 

5 

80 

8 

66 

13 

75 

15 

94 

106 

10 

80 

10 

88 

la 

75 

14 

88 

99 

a 

88 

11 

69 

13 

81 

14 

88 

96 

6 

Vf 

t 

44 

9 

56 

13 

75 

93 

3 

44 

8 

86 

9 

58 

15 

94 

83 

9 

81 

8 

5T 

6 

37 

11 

69 

80 

? 

18 

4 

36 

7 

44 

11 

69 

Average: 
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XX 

88 

ox 

08 

8 

*e 

ax 

58 

ox 

^ 

e 

03 

8 

M 

ax 

at 

sx 

r^a 

6 

oa 

3 

«8 

#x 

a? 

sx 

Sd 

OX 

oa 

a 

8?» 

>x 

xa 

KX 

ee 

XX 

aa 

e 

CY 

sx 

8a 

e 

*> 

t 

V£ 

a 

*e 

8X 

aa 

<s 

aa 

8 

^ 

Y 

68 

XI 

T€ 

8 

?s 

8 

xc 

e 

OS 

XX 

>♦ 

T 

e£ 

^ 

sx 

£ 

33 


Table  8b 

Breakdown:   Sxperlecce- — 6  Months  to  13  Tears. 
Suaber  of  Haters:   16 
Kvaluation  lor  oonsletency. 


Average 
RatlDgB 

Sequence 

Shown. 

Mo. 

±5% 

Ko.  fi 

No.  f 

±20f. 
Ko.  i 

153 

If 

.-8 

38 

li 

88 

18 

76 

15 

94 

136 

4 

m 

U 

88 

14 

88 

15 

94 

136 

U 

31 

8 

60 

13 

7S 

14 

68 

130 

a 

38 

10 

83 

10 

63 

14 

88 

133 

i 

80 

9 

66 

13 

76 

15 

94 

116 

6 

44 

11 

69 

13 

75 

16 

94 

107 

10 

80 

11 

89 

18 

76 

14 

86 

105 

8 

86 

10 

63 

18 

76 

15 

94 

105 

6 

31 

7 

44 

8 

60 

13 

81 

103 

3 

80 

U 

89 

11 

69 

16 

94 

84 

9 

35 

6 

36 

6 

38 

11 

69 

83 

7 

IB 

8 

31 

6 

38 

11 

69 

Average: 

40^ 

9 

bTi 

11 

66^ 

14 

87' 

6 


r 


tf8  9i<:f.<*r 


':^ 

.K£ 

V, 

'/.Cf 

^Ti 

J5«± 

t 

.0^ 

.  0'-' 

'.'• 

* 

,'-> 

A9 

'  ■ 

37 

;U 

91 

£i 

".'  r 

s 

4^ 

ai 

8a 

♦X 

96 

IX 

3« 

r, 

96 

♦1 

m 

ffi 

0« 

9 

15 

a 

»« 

♦X 

n^ 

OX 

gs 

OX 

a«!: 

d 

#€ 

91 

m 

fii 

da 

9 

oe 

8 

»e 

kl 

et 

6X 

-«b 

XX 

f4^ 

r 

#8 

M 

m 

iii 

e© 

XX 

oa 

8 

^ 

ai 

m 

M 

■«?5 

OX 

B8 

4 

^ 

15i 

m 

% 

-f*.^ 

V 

iffi 

a 

m 

9i 

QB 

^X 

€^ 

11 

08 

d 

m 

■CI 

^ 

1 

1^ 

% 

m 

*^ 

^ 

11 

^S 

'n 

G 

^ 

^ 

m 

t 

^^^va      H    .5^38     if    j^a     e     ^4^     8 


SX 

...3i 

* 

dsx 

XX 

8SX 

8 

.•  .i. 

^ 

Wi 

i 

Wl 

ox 

VOi 

« 

dOX 

d 

« 

f:oi 

% 

i>b 

% 

S€i 

:fcS6«^A 

m 


Table  H 

Breakdown:   Experience 3  To  4  Years. 

SuBber  of  Raters:   19. 
Svaiuation  for  aoouracy. 


Corrected 

Ratittge 

Sequence 
Shown 

4- 
So. 

Ho. 

^0 

4 

±10* 
Ho.  1L 

±20* 
Ho.  * 

155 

13 

9 

47 

14 

74 

14 

74 

19 

100 

143 

4 

5 

ae 

11 

68 

16 

84 

19 

100 

137 

11 

10 

53 

11 

58 

17 

90 

19 

100 

132 

8 

13 

63 

15 

79 

15 

79 

19 

100 

132 

X 

9 

47 

14 

74 

16 

84 

19 

100 

118 

5 

9 

47 

13 

69 

14 

74 

19 

100 

105 

10 

6 

32 

8 

42 

12 

63 

18 

95 

98 

a 

8 

42 

10 

53 

12 

63 

18 

95 

95 

6 

6 

33 

9 

47 

11 

58 

16 

84 

92 

3 

7 

37 

8 

42 

10 

63 

17 

90 

81 

9 

4 

31 

6 

32 

8 

42 

13 

69 

79 

7 

8 

16 

4 

21 

6 

32 

15 

79 

Average: 


7        39Jt     10        54*^     13        66*     18 


93-^. 


J^OSjK 

^Xi 

Hu 

•*^ 

e+ 

^ 

.o^ 

^ 

.oil 

^ 

,  o''^ 

JT 

.o^! 

00  i 

dl 

:?V 

■*x 

l^f 

*X 

V.V 

e 

001 

81 

^8 

ax 

8d 

IX 

O^' 

a 

001 

91 

oe 

vx 

3a 

XX 

sa 

ox 

QQl 

ej 

eT 

3X 

9V 

ax 

ea 

ki 

odi 

9i 

^8 

ax 

*v 

^x 

V* 

oox 

ex 

*V 

*x 

98 

sx 

V* 

8 

es 

ax 

€B 

sx 

S> 

8 

b'S 

a 

a^ 

8X 

€B 

SI 

sa 

OX 

6^ 

8 

#e 

ax 

83 

XX 

T* 

e 

S5 

d 

08 

TX 

sa 

ox 

S> 

8 

?« 

V 

ea 

oX 

s^ 

a 

£5 

a 

xs 

^ 

eT 

3X 

ss 

IS 

^ 

3X 

s 

^59       8X     J^d       5X     ^d       OX     j^«?S 


SX 

ddX 

S^X 

XX 

V£X 

8 

S£i 

I 

sax 

a 

BJX 

ox 

GUI 

s 

Be 

a 

ae 

;> 
s 

i»« 

e 

X8 

8? 

!eaATeTA 

34 


Table  9b 

Breakdown:   Sxperience 3  To  4  Yeare. 

Wumber  of  Rat era:   19. 
I^valuatlon  for  oonB latency. 


Avsrag3 
Ratings 

>tiown 

No.  it 

tn 

Ho. 

1* 
It 

No.  i 

Ho.  ii 

147 

13 

7 

87 

X8 

88 

18 

84 

19 

100 

135 

4 

10 

53 

15 

79 

18 

95 

19 

lUO 

134 

11 

9 

47 

13 

63 

16 

84 

19 

100 

131 

8 

18 

63 

m^- 

79 

15 

79 

19 

100 

131 

I 

9 

47 

15 

79 

16 

84 

19 

100 

116 

5 

11 

58 

18 

68 

15 

79 

18 

95 

111 

10 

9 

47 

18 

68 

15 

79 

18 

95 

103 

8 

a 

18 

8 

48 

10 

88 

18 

95 

102 

6 

7 

37 

9 

47 

9 

47 

15 

79 

98 

8 

4 

81 

U 

68 

18 

68 

18 

95 

83 

9 

6 

36 

8 

43 

8 

43 

IS 

68 

84 

7 

4 

81 

8 

33 

9 

47 

18 

68 

Average; 


8    39*  13   61^  13   70f=.  17   91% 


«9  eldjRA 


m^ 


f 

4^: 

'^.t 

« 

-^i             ©O:  ■ 

■  "■■ '-'"'. 

• 

Ci'V 

.o^' 

.©?? 

t  * 

■   .   •  '<Br. 

ei 

^B 

8X 

aa 

EX 

^.!: 

V 

Vbl 

as 

.Pf 

pr 

pr 

TT? 

01 

# 

dLl 

UWi. 

^;i 

El 

e 

ix 

^fiX 

001 

ex 

et 

51 

e? 

:#X 

se 

8X 

8 

xcx 

GOi 

er 

^  r 

iX 

ex  I 

&i 

ax 

8a 

? 

9 

ox 

XXX 

a« 

91 

,.  - 

-.^ 

-  i  X 

'■--* 

8i 

■  '„ 

e 

iOi 

6 

r^  — ' 

■*■ 

■  ^ 

9 

V5 

V 

• 

aox 

ae 

Bl 

sa 

51 

S3 

SX 

XS 

^ 

8 

ae 

f?. 

e 

£^ 

8 

ct 

ti 

• 

2^8 

6t 

S.C 

'  . 

e 

ss 

d 

X£ 

* 

^6 

a,':V 

Ti 

,-'0V 

■^x^ 

ax 

^K'S 

8 

:»7"ir,T«yA 

36 


Table  lOa 

Breakdown:      Experience Over  4   Years. 

Kuflitoer  of  Raters:      29. 
Evaluation  for  aocur&oy. 


Corrected 
Ratings 

Sequence 
Shown 

Ho. 

^ 

1& 

Ho. 

^ 

i 

±10* 

Ho.  f 

No.  f 

151 

13 

14 

46 

18 

63 

31 

73 

87 

93 

140 

4 

e 

31 

16 

55 

19 

66 

37 

93 

133 

11 

20 

69 

35 

86 

35 

86 

27 

93 

138 

8 

10 

35 

16 

43 

33 

76 

36 

90 

119 

1 

14 

48 

30 

69 

35 

86 

36 

90 

115 

5 

13 

45 

17 

59 

33 

83 

37 

93 

103 

10 

8 

38 

12 

41 

13 

45 

36 

90 

98 

a 

13 

45 

16 

55 

19 

66 

37 

93 

92 

8 

11 

37 

13 

45 

14 

48 

27 

93 

90 

3 

8 

38 

12 

41 

14 

48 

35 

86 

79 

9 

5 

17 

7 

24 

8 

38 

31 

73 

77 

T 

9 

31 

15 

53 

17 

59 

33 

79 

Averai-ei 


11       39^     16        53^     18        63^     36       B9$ 


,^ 

.Q^ 

^ 

.o-f 

> 

.  or 

k 

.  C" 

. '  irf 

«iiOJRMfI 

£6 

\   ■• 

&\ 

18 

^& 

<il 

3^ 

*X 

Si 

idl 

m 

T?: 

af» 

91 

^"T* 

31 

IS 

0 

XI 

i;G 

di 

3S 

Oi 

« 

(^.1 

o«^ 

dS 

e 

r; 

ee 

OP 

t?^ 

3^1 

t 

OAJ. 

<^ 

Tg 

V  v5 

5R 

9a 

Vl 

«*■ 

a 

-IS 

axx 

p« 

fcS 

.i* 

W 

x# 

S«f 

as 

8 

Oi 

SOI 

ee 

vg 

a« 

ex 

23 

8£ 

a^ 

51 

€ 

8^ 

^ 

J^ 

r>* 

T'A 

a^* 

U 

^£ 

i:i 

i 

se 

68 

as 

e> 

*i 

x^ 

Si 

s 

« 

oe 

SV 

iB 

8S 

8 

»8 

V 

Vi 

a 

« 

9T 

«t 

£g 

«3 

VX 

sa 

ai 

i€ 

e 

t 

VV 

68 

ds: 

if^d 

di 

^Cc 

o  J 

a 

.••^^•^A 

86 


Table  10b 

Breakdown:   Sxparience— Over  4  Yeare. 
Number  of  Raters:   39 
Evaluation  for  Qouaiut&uoy. 


Average 

Hat IngB 

iSequenoe 
Shown 

4> 

Ho. 

9lL 

Ho.  f 

tio^ 

Ho.  i 

±30* 
So.  i 

1%5 

IM 

13 

%5 

30 

69 

88 

79 

87 

93 

133 

4 

U 

66 

88 

76 

84 

88 

88 

97 

153 

11 

80 

S9 

85 

86 

85 

86 

27 

93 

126 

• 

IS 

46 

18 

68 

19 

66 

36 

90 

117 

1 

IS 

86 

88 

76 

88 

76 

86 

97 

113 

S 

n 

86 

18 

68 

81 

78 

87 

93 

108 

10 

9 

81 

18 

46 

19 

66 

m 

86 

99 

8 

II 

38 

18 

45 

19 

66 

87 

98 

97 

6 

u 

88 

17 

59 

19 

66 

87 

98 

95 

8 

10 

86 

13 

45 

18 

68 

86 

90 

87 

9 

10 

36 

II 

88 

14 

48 

88 

79 

82 

7 

14 

48 

14 

48 

14 

48 

88 

76 

Average: 

IS 

44^ 

17 

591^ 

30 

67^ 

36 

90 

as 


tfOl  9ida7 


^OHi- 


>       .OM 


8&         dV         jfcS 


,3lA©'^     ^    10  VQ Cvj.it?i 


■  XO 


Visi  Q&  ^0  *b  rt\  Sfci 

58        VK        m       afi       88        riS 


a\         Sfo 


^e      Ts      €v      IS      ga      si 


9V 

&^ 

B*>> 

#X 

8t 

£'• 

8A 

H 

8^ 

CO 

8€ 


''30 


3^ 


m 


se 

Tg 

d8 

..  ; 

o** 

cii. 

Bi, 

ii 

se 

VR 

»a 

ex 

^a 

n 

se 

11 

Oi 

^1 


-i  is  Jii-':^.     IC      l&QiLUtl 


U 


•I;,  a. 

I 


e 


tlX 
SIX 

tiul 
GQ 


;^09 


>*        Si 


ST 


Tabl«  11a 

Brea&dovm:   Education Initial  Tiait  Study  Training  Received 

In  College. 

Number  of  Raters:      22. 

HIvaluation  for  accuracy. 


Correoted 
Ratings 

tjequenoe 
Shown 

Ho. 

516 

Ho.   ^ 

±1011 
Ho. 

1^ 

Ho. 

i 

156 

12 

13 

59 

15 

68 

17 

77 

19 

R7 

144 

4 

8 

36 

13 

59 

16 

73 

20 

91 

138 

11 

9 

41 

12 

55 

15 

68 

18 

83 

133 

8 

9 

41 

13 

55 

14 

64 

17 

77 

132 

I 

10 

46 

15 

6B 

15 

68 

2C' 

91 

119 

8 

11 

50 

13 

59 

14 

64 

19 

87 

106 

10 

9 

41 

11 

50 

IS 

59 

18 

B2 

99 

a 

8 

36 

11 

50 

14 

64 

20 

91 

96 

8 

7 

33 

7 

38 

11 

50 

16 

73 

93 

8 

6 

37 

7 

32 

7 

32 

18 

82 

82 

9 

8 

8 

5 

23 

5 

2? 

15 

68 

80 

7 

6 

33 

8 

36 

9 

41 

14 

64 

Average: 

8 

37lt 

11 

491^ 

13 

581?. 

18 

81' 

T5 


4ii    dICfAT 


b9Vi906A  ^niaiatl   t^uic^  9ml 


^--4 


S 


vv   vx 


8B    81 


*-cJ 


i^a 


}r^^f 


-^;« 


9^ 


Si 

ax 


V 


a 


J^XS   8X  «^a   51  <^»   li 


J^ 


so 


86   ei   ea   51 


^■x   =!?:   8 


ip 


UT 


IX 

e 

8 

9 

J? 

a 


avoHB       a^^iii^Bh 


61 

88X 

1 

»1 

L.i. 

8fiX 

8 

S5X 

k 

nEX 

'8 

9iX 

OX 

90X 

8 

«6 

h 

88 

s 

se 

@ 

88 

T 

oe 

:?.ii:.iii#VA 

38 


Table  lib 

BreakdoTim;      Education — -Initial  Time  Study  Training  Received 
In  Oolleg«. 

Hunber  of  Raters:      33 

SralURtion  for  oonsistenoy. 


Average 
Ratings 

Sequenoe 
3110  wn 

Ko.  i 

No.   lb 

Ho. 

i 

£30^ 

Ko. 

i 

155 

13 

11 

80 

16 

78 

17 

77 

19 

m 

138 

4 

18 

58 

18 

88 

18 

68 

19 

66 

138 

11 

9 

41 

13 

88 

18 

66 

19 

66 

131 

• 

S 

88 

11 

80 

18 

69 

17 

77 

131 

X 

y) 

46 

18 

66 

17 

77 

30 

91 

113 

8 

? 

83 

18 

89 

18 

59 

30 

91 

110 

10 

13 

S8 

13 

89 

16 

78 

18 

83 

103 

3 

5 

38 

18 

86 

18 

68 

30 

91 

101 

e 

? 

83 

10 

46 

13 

65 

16 

78 

99 

s 

10 

46 

11 

80 

18 

66 

17 

77 

87 

9 

4 

18 

6 

37 

11 

SO 

16 

73 

83 

T 

7 

83 

8 

36 

11 

SO 

16 

78 

Avarat^;e: 

• 

3»)t 

13 

bAi 

14 

64^ 

18 

83 

,S-  it 

^<^^^i  ♦PM  **T*  *i         twit/.'  tl^eWTA 

sx  sat 

*  8£X 

8  .r-^i 

1  XSiX 

dr  SXX 

01  OIX 

S  SOX 

a  xoi 

e  T8 

V  60 


jx-»  *.«■)▼  A 


j^ 

.oK 

* 

.oW 

> 

,air 

» 

.«»» 

i^ii 

u>.C 

vv 

Vi 

v^V 

dX 

g<s 

*i. 

88 

ex 

8<} 

ax 

Sd 

ax 

aa 

SX 

d8 

91 

SB 

?a 

aa 

sx 

X* 

8 

vv 

Vj. 

^cJ 

^i 

v^d 

XX 

5K 

3 

X9 

OS 

vv 

TX 

8d 

ax 

d> 

OX 

X9 

OS 

ea 

5X 

p.? 

u 

8S 

V 

^3 

3X 

ST 

ax 

ed 

Kl 

aa 

sx 

X9 

OS 

as 

ai 

53 

SI 

ss 

a 

5V 

ar 

-1 

^r 

Pa 

OX 

CT 

V 

TT 

vx 

B<> 

'■  X 

^t'C, 

LI 

cii* 

ox 

5T 

9X 

ca 

iX 

vs 

& 

8X 

* 

ST 

ax 

oa 

i£ 

-?; 

8 

se 

1 

C8 

81 

?*ika 

>J 

&.i 

*^tfB 

B 

39 


Table  13a 

Breakdown:      Education — ->Inlljial  Time  Study  Training  Received 
lith  tlie  Company. 

Number  of  Raters:     45. 

Evaluation  for  accuracy. 


Corrected 
Ratings 

Sequence 
Shown 

Id.  ^ 

No.   1^ 

Bo.  i 

i30^ 
Ho.  <^ 

153 

13 

23 

49 

30 

67 

33 

71 

43 

95 

141 

« 

17 

38 

39 

64 

31 

69 

43 

93 

135 

11 

35 

55 

35 

78 

41 

91 

43 

93 

130 

• 

35 

56 

33 

71 

39 

87 

43 

93 

131 

1 

30 

44 

31 

69 

37 

83 

43 

95 

117 

S 

18 

40 

30 

67 

31 

69 

43 

93 

104 

10 

13 

37 

30 

44 

36 

58 

40 

89 

S7 

8 

19 

43 

34 

53 

31 

69 

40 

89 

94 

6 

14 

31 

19 

43 

81 

47 

36 

80 

91 

3 

14 

31 

33 

49 

35 

55 

36 

80 

60 

• 

13 

37 

14 

31 

19 

43 

30 

67 

76 

? 

7 

16 

17 

38 

31 

47 

34 

76 

Average: 


16       Z&iL     34       58# 


66^     39       B7f 


ec 


^-i 

m± 

■■■#fttt 

^m 

'# 

»OTT 

> 

,0^^ 

>, 

.OW 

^ 

.<^ 

ii^ 

XV 

s^ 

?8 

05 

d» 

ss 

>« 

«# 

ea 

i£ 

J^ 

8S 

as 

fX 

se 

8» 

i« 

1* 

8V 

an: 

86 

as 

59 

S* 

V8 

ee 

IT 

El 

aa 

86 

t€ 

€^ 

Id 

^c 

€a 

i& 

#^ 

OS 

ee 

s* 

Od 

X5 

V8 

0£ 

81 

.t» 

Uf 

8d 

ae 

«^ 

06 

?ti 

sx 

^ 

0# 

m 

iC 

«a 

^B 

8^ 

ex 

08 

as 

T* 

15? 

8^ 

Ql 

xs 

^x 

06 

8"^ 

«e 

as 

ejk 

8S 

i« 

♦X 

T8 

05 

s^ 

91 

i& 

J^X 

Vl?. 

sx 

m 

*£ 

t^ 

IS 

d£ 

Vi 

ai 

V 

>^       m     ^8        8?,;      ^c        H:     dm 


mi 

sax 

■f 

x^x 

JL± 

aax 

8 

Ofil 

'I 

XftX 

m 

TXX 

ox 

^x 

:i 

re 

m 

^« 

'9 

xe 

'0 

06 

'^t 

at 

t^i^HTM 

40 


Table  13b 

Breakdown:   Education Initial  Time  Study  Training  Received 

With  the  Company. 

number  ol  Raters:   45 

Evaluation  for  consistency. 


Average 

Ratings 

r^equenoe 

Shown 

Bo.   ^ 

No.   i 

^1016 

So.  i 

480^ 

147 

13 

20 

44 

33 

71 

34 

75 

43 

95 

135 

4 

31 

47 

33 

73 

41 

91 

43 

95 

134 

11 

27 

60 

34 

75 

40 

89 

43 

93 

131 

8 

23 

51 

33 

71 

35 

78 

42 

93 

122 

1 

20 

44 

38 

63 

36 

78 

43 

95 

118 

5 

18 

40 

30 

68 

33 

73 

42 

93 

110 

10 

20 

44 

SO 

67 

35 

78 

40 

89 

104 

a 

14 

31 

33 

49 

38 

63 

40 

89 

102 

• 

30 

44 

35 

55 

35 

55 

37 

83 

99 

8 

30 

44 

33 

61 

39 

64 

39 

87 

87 

9 

17 

38 

18 

40 

35 

55 

33 

73 

85 

7 

X3 

37 

17 

38 

34 

53 

33 

71 

Average: 

19 
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37 

601^ 

33 
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40 

86 
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> 

,m 
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3^ 

s> 

3T 
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IK 
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ae 
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se 
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u. 

:-^ 
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» 

XI 
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s» 

8T 

38 

£V 

ss 

iS 
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»^ 

«* 
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?^. 

^^ 
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0^ 
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SG 

s^ 
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5^ 

Q'. 
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0« 

0^ 

ft^ 

95 

T9 

0« 

^ 

OS 

ox 

01.  X 

©B 

r.L 

c?? 

PO 
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?i^ 

4f 

AOiX 

S- 

r. 

dOi 

V8 

95 

>d 

es 

X 

0t 

rt 

;•*• 

;»;» 

ff? 

0.M 

Q.s' 

"^y 

ft 

j  • 

'.] 

GS 

i>€ 

8£ 

VI 

vs 

SI 

39 

j^8        0»     4fXV        S£     JH^        VB      ^^        «i  >«a««»TA 
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Table  13a 

Breakdown:      Rater  Uses  Hla  Own  Concept  of  Btandard   Perforraar.ce. 
Number  of  Raters;      54 
iTaluatlon  for  accuracy. 


Corrected 
Ratings 

Sequence 
Shown 

No. 

.* 

* 

Vo.  % 

Ho.   f 

Fo.  i 

153 

13 

34 

44 

32 

59 

35 

65 

61 

94 

142 

4 

34 

44 

35 

46 

33 

61 

50 

93 

136 

11 

27 

50 

36 

67 

45 

83 

50 

83 

130 

8 

33 

43 

35 

65 

43 

80 

49 

91 

121 

I 

34 

44 

36 

67 

43 

80 

49 

91 

117 

6 

34 

44 

37 

68 

36 

70 

51 

94 

104 

10 

13 

33 

23 

43 

30 

55 

49 

91 

97 

8 

20 

37 

28 

58 

37 

68 

48 

89 

94 

6 

17 

33 

21 

39 

33 

43 

43 

80 

92 

Z 

19 

35 

33 

41 

2B 

63 

47 

87 

80 

9 

IS 

33 

16 

30 

23 

43 

38 

70 

78 

f 

13 

33 

20 

37 

24 

44 

38 

70 

Average: 


30 


37-^     38        51*^      34        63f      47        87f 


u 


Jl&l    9 


.9: 


■  'St 

•  ■  I 

3^   -  0^ 

.oK 

ovoJS 

i       TOO 
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-t 

38 

35 

83 

i:Z 

>i^ 

*s 

81 
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£9 

03 

13 

KS 

::  ^ 

as 
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» 

,-. .  f 

e9 

oa 

sa 

a^ 

va 

es 

oe 

V8 

II 

id 

6* 

08 

5* 

59 

65 

5* 

5S 

3 

U5I 
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.-.. 
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i'^ 

■>S 

^ 

I 
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*e 

id 

OT 

dl 

i 

T5 

»j^ 

>£ 

a 

VXI 

le 

e> 

3c 

£S 

'"S 

SI 
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s 

* 

>0I 
V9 

V3 

5:?^ 

"Jf^ 

ss 

0  f 

8 
5 

>9 
59 

ov 

•s  ', 

';■• 

ax 

eg 

^-ii 

8 

0? 

8^ 

;^«» 

^£ 

v.^ 

OS 

ss 

51 

t 

V 

8V 

Vi^ 

.,  X' 

0 '  ■ 

V  ■ 

^'.' 

:9^jai©vA 
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Table  13b 

Rreaitdown;      Rater  Uses  His  Own  Ooncept  of  .'Standard   Perf orfaarice, 
Sfumber  of  Raters:      54 
Evaluation  for  consletency. 


AverE 

Rati  J 

lg»3 

Semen  ce 
':hown 

no.     ^ 

No. 

1^ 

iia^ 

V.o,      i 

+20^. 
IIo.      i 

147 

12 

17 

33 

31 

57 

39 

73 

50 

93 

134 

4 

24 

44 

34 

63 

46 

85 

52 

93 

134 

11 

^ 

53 

39 

72 

44 

81 

50 

92 

ise 

8 

16 

30 

27 

50 

43 

80 

49 

91 

120 

I 

37 

50 

32 

59 

45 

80 

49 

91 

114 

5 

88 

5S 

35 

65 

40 

74 

51 

94 

108 

10 

18 

33 

26 

46 

36 

67 

48 

89 

102 

2 

30 

37 

30 

56 

34 

63 

48 

fc9 

IDS 

6 

81 

39 

30 

56 

30 

56 

42 

78 

ee 

Z 

16 

30 

30 

56 

37 

68 

47 

87 

85 

9 

14 

36 

18 

33 

37 

50 

h2 

78 

83 

7 

13 

24 

14 

36 

19 

35 

35 

65 

Average: 

20 

38^ 

as 

53  V. 

37 

SB-^ 

47 

87' 

dU  fldmr 


•»^1   iu'i 


?..n 

Pi-i- 

•' .  : 

vi«* 

m 

m 

fit' 

i»v 

^^ 

1 

} 

.^m*                ^{^X*-                 ^l'^¥  ''^-".4  c.-::^:---.  ,    -r-Nva 

.^y      ^^  iJ  HI 

ft^^        ^ii                       ':        Ar!        0>'  ^^:        D-  ,S  ---- 

iiV        &«>  dd        \io        ad        0*  _d  ^>0i 

■13      do  vi'-:      !?i      ■:'!^i      ^i  #s      ^i              V  ^a 


43 


Table  X4a 

Breakdown:   Rater  Uses  3oae  Film   or  Otncr  l^nbodiment  of 
Standard  Perforaianoe. 

Huaber  of  Haters:   15. 

BTvaluation  for  aoouracy. 


Correct*Jd 
Ratings 

Sequence 
Shown 

Ko.   i 

Ho.   1^ 

±xojl 

No.   ^ 

4^* 

Ho.  f 

155 

13 

8 

53 

14 

93 

15 

IOC- 

i5 

100 

143 

4 

4 

37 

8 

53 

14 

93 

15 

100 

137 

li 

11 

73 

11 

73 

14 

93 

15 

100 

131 

8 

9 

60 

13 

80 

13 

80 

15 

100 

133 

X 

8 

53 

10 

67 

13 

80 

lb 

100 

118 

B 

5 

33 

11 

73 

13 

87 

15 

100 

104 

10 

6 

40 

8 

53 

11 

73 

15 

100 

98 

2 

10 

67 

11 

73 

13 

87 

15 

100 

95 

6 

8 

53 

9 

60 

11 

73 

15 

100 

93 

S 

5 

33 

7 

47 

8 

53 

14 

93 

81 

0 

2 

13 

3 

20 

4 

27 

13 

87 

79 

? 

4 

37 

6 

40 

8 

53 

14 

93 

Average: 

7 

44^ 

9 

61^ 

11 

751i^. 

15 

98 

JB* 


1q  tafioiibodit'l  tou^O  to  mill  aeoi^  e»sU  i»^«H      ::f.«ob3U8^a 


.clX      :8i«^«JI  to  ledsjuK 


^   .Off 

001 

c 

ax 

ex 

£6 

H 

sa 

8 

£X 

adx 
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a/ 

i.e 

#x 

Sd 

9 

c*«r 

1,  ■  ■ 

^ 

..J  .1 

04)1 

ai 

se 

^x 

6? 

iX 

5V 

Li 

V£X 

001 

ai 

oe 

BX 

08 

8{ 

06 

8 

i5 

x«x 

.J--I 

■■ 

J. 

V3 

*:;X 

sa 

e 

i 

S&I 

001 

ai 

T8 

£i 

£V 

iX 

€fi 

a 

a 

8XX 

001 

ei 

«Y 

XX 

^ 

e 

o> 

p 

ox 

K)X 

vw**. 

dl 

V6 

6^ 

..    5 

-  '- 

-  i 

8 

89 

001 

ax 

£V 

XX 

oa 

e 

•*;a 

s 

a 

ae 

se 

>x 

sa 

8 

v# 

V 

iiS 

a 

c 

se 

Vo 

- 

YS 

> 

OS 

s 

sx 

8 

« 

Xd 

se 

^x 

sa 

8 

0* 

a 

VK 

^ 

V 

^x 

►89 

ax 

>eY 

iX 

^xa 

e 

H* 

V 

;e;i/?i«vA 
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Table  14b 

Breakdown:  Rater  Uses  Sone  Film  or  Otner  Embodiment  of 
v^tpndard  Performance. 

number  of  Haters:   15 

ETaluatlon  ior  oonsist'^ncy. 


Avera^^e 

Ratings 

^Sequence 
Shown 

iro. 

8* 

Ko.  iL 

No.   # 

Ko.  i 

150 

13 

13 

80 

15 

100 

15 

100 

15 

iOC 

13b 

4 

13 

80 

13 

87 

14 

93 

15 

100 

135 

11 

10 

67 

11 

75 

14 

93 

15 

100 

129 

8 

10 

67 

10 

67 

11 

73 

15 

100 

119 

I 

9 

60 

11 

73 

14 

93 

15 

100 

117 

5 

5 

33 

10 

67 

13 

80 

15 

100 

111 

10 

9 

60 

15 

100 

15 

100 

15 

100 

100 

3 

7 

40 

13 

87 

14 

93 

15 

100 

99 

6 

8 

53 

9 

60 

18 

80 

15 

IOC 

99 

Z 

10 

87 

11 

73 

13 

87 

15 

100 

87 

9 

4 

27 

5 

33 

9 

60 

13 

87 

63 

f 

4 

37 

7 

40 

9 

60 

14 

93 

Average: 

8 

56^ 

11 

72-r 

13 

84^> 

15 

98< 

/«- 


lo  timmltodaSL  itn^O  lo  wlf?  ©snc^*^   tsirl!  isifiil?      .fr^rr.rf-a-tf? 


tnr.4.  V'j^.  ^*./-r.^ 


.:ioj[    aj      vq      si      ^r      x 


i        08        «•! 


ax     ^ft*  /jt  te  CI  "T?^  dj[ 

ooi    f^i      OB  <jx  vy  ox  Kc  a 

OCX  31  t-vni  tl  OB  ^15 

OCX    ax      08  81  OB  I?  sa  a 


r  r>' 


X  IX  a'e'x 


^9      ax    >^B      ^i  XX    >aa      8 


"  r 


§ix 


i^  001 


a 

ee 

% 

«<} 

6 

^ 

T 

^^ 

t9r^i»VA 
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Table  15a 

Breakdown:      ^lant   employe f>B  nuiiber  leee  than  101. 
Number  of  raters:      10 
Valuation  for  accuracy. 


Corrected 

Rat^.Tif^e 

i^ftouence 

Wo.  4 

Wo. 

tioit 

Vo.  <^ 

±aoi 

Ko.   ^ 

14t> 

13 

Z 

30 

5 

50 

5 

50 

9 

90 

135 

4 

3 

30 

4 

40 

8 

80 

10 

100 

139 

11 

5 

50 

7 

70 

7 

70 

10 

100 

134 

S 

5 

50 

5 

50 

7 

70 

10 

100 

115 

1. 

7 

70 

7 

70 

9 

90 

10 

100 

111 

6 

7 

70 

7 

70 

7 

70 

9 

90 

99 

10 

2 

80 

3 

SO 

5 

50 

9 

90 

93 

2 

5 

50 

7 

70 

8 

80 

10 

100 

89 

6 

0 

0 

0 

0 

1 

10 

7 

70 

87 

3 

3 

30 

3 

80 

3 

30 

9 

90 

77 

0 

1 

10 

1 

10 

3 

30 

5 

50 

74 

? 

1 

10 

1 

10 

1 

10 

6 

60 

Averf?g«: 


33^. 


4li 


53<       9        BH 


a^ 


.lOji    .,^^aj  sasX   i^dmu'/T   •"—■ '"of  :.*iw.>'?x'j'r3x-:i 

,XOjaiwoo.e  tot  acJ!:isuiaiT7 
.>OSj^  ^i^^  ^▼^  -^'i         liOttBury^U  ^1»?o•97•:or> 


e 
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d 

Od 

Q 
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& 
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03 

3 
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f. 
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QX 

OV 
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1 
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9 
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09 
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OV 

T 
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09 

e 
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a 

OS 
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ox 
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8 

Ov 
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CI 
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i 

vx 

? 

^T3      ??      ^sa      ^      ^r.p-      -^.      ^'^^: 
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8 
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X 
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a 
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ox 
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TY 

T 

i.y 

:©:g6i9vA 
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Table  15b 

Breakdown;   Plant  e«ployecp  number  less  than  101. 
Number  of  raters;   10 
Evaluation  ior  consistency. 


Average 

Ratings 

Sequence 
Shown 

No.   ^. 

ISO, 

^ 

^ 

±10^ 

No.  IL 

±aof. 

Ko.  f. 

146 

13 

2 

30 

6 

feO 

5 

50 

9 

90 

127 

4 

6 

60 

9 

90 

9 

90 

10 

IOC 

130 

11 

b 

50 

7 

70 

7 

70 

10 

100 

133 

a 

5 

50 

5 

50 

7 

70 

10 

100 

117 

I 

4 

40 

8 

80 

B 

80 

10 

100 

106 

6 

5 

50 

6 

60 

8 

80 

8 

80 

103 

10 

2 

30 

6 

60 

6 

60 

9 

90 

98 

a 

5 

50 

7 

70 

7 

70 

10 

100 

91 

6 

0 

0 

0 

0 

3 

30 

7 

70 

91 

z 

2 

^ 

4 

40 

5 

50 

9 

90 

78 

9 

1 

10 

2 

30 

3 

30 

5 

50 

77 

? 

1 

10 

3 

30 

4 

40 

6 

60 

Average: 

3 

33*^ 

^ 

534 

6 

601^ 

9 
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OS 
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6 

8V 
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05 
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6 
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^ 

^^t. 

e 

:©j^ievA 
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Table  16a 

Breakdown:      T'lant  eaployecs  number  101-1090, 
TTumber  of  rat  ere:      37. 
^Valuation  for  accuracy. 


Corrected    -'ecuence 
Ratings       Shown 


153 

13 

141 

4 

135 

IX 

130 

8 

130 

1 

116 

5 

103 

10 

97 

3 

93 

6 

91 

8 

80 

9 

78 

7 

Average: 

i- 

54 

±^^ 

^4 

±10* 

±20^ 

m. 

4 

!?o. 

< 

?^o. 

^ 

Fo. 

^ 

ao 

54 

34 

35 

36 

70 

34 

93 

IS 

41 

88 

63 

85 

68 

33 

84 

30 

54 

87 

78 

80 

81 

33 

84 

15 

41 

33 

60 

39 

78 

33 

87 

30 

54 

85 

68 

81 

84 

33 

84 

19 

51 

85 

68 

86 

76 

35 

95 

6 

16 

13 

35 

15 

41 

34 

92 

1« 

41 

30 

54 

88 

76 

34 

92 

1ft 

41 

1ft 

48 

19 

51 

38 

87 

1ft 

41 

80 

54 

88 

63 

30 

ei 

10 

87 

11 

80 

16 

43 

88 

76 

ft 

n 

17 

46 

30 

54 

^9 

78 
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av 
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^i 
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^^ 
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..^■■i 
^^. 

V3 

S5 
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■■ 

i# 

3X 

fi^ 

5^ 

H^ 
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^^ 
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^X 

5 
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^ 

ai 

-■ 
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Oi 

9 

08 

rv 

'O 

>a 
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TX 
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^ 

^^r 
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■ 

3a 

vf 
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■~  f;.^ 

-71 

TT  1 
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Table  16b 

Breakdown:      Plant   emplo/ees  number  101-1000 
Kuffioer  of  raters:      cl , 
Evaluation  i or  consistency. 


Average 

Ratings 

Seouence 

Gnortn 

15^ 
No.   ^ 

No. 

1^ 

±10% 

Ho.  i^ 

+30^ 
No.   % 

1%4 

12 

lb 

41 

30 

54 

39 

78 

34 

92 

134 

4 

19 

51 

34 

65 

31 

84 

35 

95 

133 

11 

24 

6b 

38 

76 

31 

84 

32 

89 

\m 

8 

11 

30 

19 

51 

39 

78 

32 

87 

119 

1 

31 

57 

38 

76 

31 

84 

33 

89 

117 

6 

16 

43 

25 

68 

37 

73 

35 

95 

109 

10 

14 

S8 

31 

57 

36 

70 

33 

87 

100 

3 

16 

43 

31 

57 

37 

73 

34 

92 

101 

8 

14 

38 

19 

51 

23 

62 

31 

84 

96 

3 

16 

43 

18 

49 

33 

63 

31 

84 

85 

» 

9 

34 

11 

30 

19 

51 

28 

76 

82 

T 

16 

43 

17 

46 

18 

49 

26 

70 

Average J 

16 

43^ 

31 

57^ 

36 

71^ 

32 

86< 

lKm± 

>exjK 

i 

^^^^i 

^ 

5i 

\ 

.yH 

,on 

■'i 

.OH 

*; 

.Ol{ 

se 

:^5 

B'^ 

^ 

^d 

OS 

IP 

di 

dt^ 

as 

i-h 

ii 

de 

>£ 

id 

Of 

S3 

Si 

#a 

IS 

ev 

8K 

dc? 

^H 

V9 

ss 

av 

es 

15 

Gi 

Oi^, 

XX 

9B 

?;-^ 

#B 

I?: 

8%- 

PH- 

Vd 

IK 

de 

as 

sv 

vs 

ar^ 

as 

:^* 

8X 

^•8 

ss 

ov 

ag 

V3 

IS 

8« 

^X 

S9 

^s 

€V 

vs 

Vd 

it 

^.i^ 

dX 

i'B 

ii; 

sa 

£9 

^  *'; 

■*  v.. 

■■'  >'., 

^x 

^S 

xs 

Sd 

o'-fr* 

ii 

o> 

ax 

av 

as 

iQ 

tvl 

■<JK.t 

li 

iss 

*j 

ov 

ac 

£^ 

>3I 

3:^ 

VI 

ia^ 

ax 

6X  >M 

II  SfiX 

B  9f*X 

a  Txx 

01  90  X 

3  XOX 

5  ae 


^ftjR 


ae      85    ^XT      as    ^ve      xs    ^s^      ex  :»:',.vi»va 
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Table  17a 

Breakdown:   '^lant  eaployeei;  number  greater  than  1000. 
Huaber  of  rst^rs:  21 
Valuation  :  or  aocuraoy. 


Oorrected 

Ratings: 

Seouenoe 

KO.  '^i 

£7K 
No.   ^. 

+  10^ 

No. 

156 

la 

12 

57 

14 

67 

16 

?6 

21 

100 

144 

4 

7 

53 

13 

57 

16 

76 

21 

100 

138 

11 

9 

42 

12 

57 

17 

81 

20 

96 

132 

8 

13 

57 

15 

73 

15 

78 

21 

100 

133 

X 

10 

48 

13 

62 

14 

67 

21 

loor. 

119 

5 

xa 

57 

13 

63 

14 

67 

21 

IOC 

106 

10 

11 

54 

15 

73 

16 

76 

19 

91 

99 

3 

9 

43 

10 

48 

14 

67 

18 

86 

96 

6 

10 

48 

13 

57 

12 

57 

16 

76 

93 

S 

6 

39 

8 

38 

9 

4o 

19 

91 

82 

9 

4 

19 

7 

33 

7 

33 

14 

67 

79 

7 

1 

6 

5 

24 

7 

33 

16 

76 

Aver?iiT«j 

9 

4lf 

11 

54^ 

13 

63'* 

19 

90f. 

bVI   bI-aI 


OuOi  iiadJ"  iB^RSTT] 


T   Jo   tp>(imttM 


vr^-j-*- 


^14' 


MVf 


>S5 


'?«  +  n  Tfoifjj'Srfiir     N*  .^ 


wx 


mi 


r\r\  r 


W-'A 


'^X 

8 

v!.X 

I 

ipcx 

o 

•^u 

c)OX 

5 


o 


5G 


ev 


:«7:»pt«»v4 


Table  17b 

Breakdovn:   Plant  aaployees  auabex  greater  tnan  1000. 

ii  u.; . .  - '  of  raters:   21 
Evaluation  for  ooi^sistenoy. 
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Average 
Eatings 

dequenoe 

bhown 

Ho.  % 

Mo.  i 

4x 

So. 

i 

Ho.  f 

149 

12 

13 

62 

14 

67 

30 

9U 

21 

100 

136 

4 

14 

67 

17 

81 

ao 

96 

21 

100 

135 

11 

10 

48 

13 

6;;^ 

20 

96 

SI 

100 

120 

8 

11 

53 

15 

72 

16 

76 

21 

100 

132 

1 

10 

48 

13 

6'd 

14 

67 

21 

100 

117 

5 

9 

<k3 

13 

57 

14 

67 

20 

96 

108 

10 

10 

48 

15 

T/i 

17 

81 

19 

91 

106 

3 

6 

a9 

11 

53 

14 

67 

19 

91 

104 

6 

7 

33 

la 

5? 

13 

62 

18 

86 

101 

3 

12 

57 

13 

63 

17 

fal 

30 

96 

88 

9 

7 

33 

11 

51'. 

12 

57 

18 

86 

85 

7 

6 

29 

9 

43 

11 

53 

16 

76 

Avers^re: 

10 

^si^ 

13 

6S^. 

16 

75^ 

20 

9^ 

d?r  aXdiiT 


OOOX  tuafit  *L9t 


•,.H 


OOX      18 


Vi3        PI 


i^d 


U 


^  c 


dd 


^e 

i:x 

11 

.■a 

e 

IS      :8T»'#3l'  to  rr? 


'j,;/.u*j..- bi  ;i  ,U;j     ^^iji     ..iiJ-i.  *.'riW.' llf  Viii 


*cil 


se 


Bi        £S 


vil 


JOX 

J.  3 

a^ 

0^ 

i»a 

CX 

e> 

OX 

»./  { 

(^ 

'»V 

'^r 

f?v 

,4  r 

5a 

IX 

«50X 

-LO 

'<i 

■^jt 

.'iu 

jju 

a@ 

OK 

va 

#^X 

Td 

toi 

Gtfci' 

i« 

.'^ 

<,'  ■ 

f^I 

«ft 

Oi 

i-i 

.1 

't^* 

d 

es 


f  r 


d 


sx 

XX 
d 

I 

a 

OX 


«? 
? 


9#i 

dSX 

vxx 

BOf 

xax 


df*0     og    :rav     ax    ^h: 
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Tnble  18a 

BreaKdown:      Sire  of  torn  Iftss  than  5000. 
iffumber  of  raters:      17 
■p^valuation  for  aocuracy. 


Corrected 

Ratings 

3eouenoe 

'shown 

t5< 
Ho.  % 

ISfo.  1L 

±90% 

T!o.  i 

150 

13 

14 

83 

15 

88 

15 

B8 

17 

100 

139 

4 

8 

47 

13 

77 

14 

83 

17 

100 

13o 

ii 

13 

71 

14 

83 

16 

94 

17 

100 

138 

8 

4 

84 

8 

47 

13 

75 

16 

94 

119 

1 

13 

77 

15 

88 

15 

88 

17 

IOC 

114 

5 

10 

59 

13 

75 

15 

88 

16 

9^. 

102 

10 

3 

18 

5 

'^9 

5 

29 

16 

94 

95 

3 

6 

47 

10 

59 

15 

88 

17 

IOC 

92 

6 

6 

35 

6 

35 

6 

35 

16 

94 

90 

S 

6 

35 

13 

71 

13 

75 

18 

94 

79 

9 

4 

24 

6 

25 

6 

35 

13 

71 

77 

7 

7 

41 

10 

59 

11 

65 

14 

85 

Average: 

8 

47<, 

11 

6?'^' 

IP, 

71^ 

16 

94' 

la 


QQi 


*;p 


■  <?  r  o  f--f  tT 


-^'1  1^    r       ^  r^      A  •!    kC 


':%& 


.x> 


JK^^'i  JfOx*  JTin  >aj^ 


»i 


83 

dX 

5G 

{;8 

• 

V 

v<9 

f^J 

r. 

/^ 

^A 

'»\ 

*s 

88 

ax 

88 

ax 

vv 

^x 


T 

io  r^dmuM 

lot  iioi:rjBijXj8var 

1 

Si 

oai 

# 

^A*;'  8i  r('' 

**;f         iiX         L'^ 

oox 


£X  X  9XX 

^  *X4 


>A'^.        -r      *^rv 


<rf 

teii 

<i 

&i 

OX 

ax 

U3 

OX 

V* 

d 

s; 

cv  ■'.. 

a 

as 

d 

' 

9 

(  .'. 

Si 

XV 

hi 

C'-* 

o 

^ 

'.i!> 

c5 

ti>: 

^s 

* 

8 

9T, 

0. 

\ 

V 

vv 

•  •■  -4 

i: 

■"^ ''  i~ 

O 

:s;.^if»vA 
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Table  18b 

Breaid07.'ri:      Size  of  toTvn  leas   than  50(;0. 
Hu»ber  of  raters:      17. 

Evaluation  for  conaietoncy. 


Average 

Ratings 

Sequenoe 

Shown 

Ro.  ^ 

Vo. 

410% 

No.   ^ 

Ko. 

146 

12 

11 

65 

15 

m 

15 

88 

17 

100 

134 

4 

9 

53 

12 

VI 

IS 

77 

17 

100 

134 

i-l 

12 

71 

14 

83 

15 

86 

17 

100 

128 

8 

4 

34 

8 

47 

13 

77 

16 

S4 

119 

1 

13 

77 

15 

m 

15 

8P 

17 

100 

112 

6 

7 

41 

11 

65 

14 

8? 

16 

94 

109 

10 

7 

41 

11 

65 

14 

83 

16 

9^; 

98 

a 

11 

65 

13 

71 

14 

85 

17 

100 

93 

s 

6 

35 

6 

35 

8 

47 

16 

94 

90 

3 

6 

35 

12 

71 

13 

77 

16 

94 

82 

9 

6 

35 

9 

53 

10 

59 

13 

77 

77 

T 

7 

41 

10 

59 

11 

65 

14 

83 

Avera.;;,e: 

8 

491^ 

11 

66fc 

14 

81^' 

16 

94 

d8i  •litisf 


8SX 
91 X 

alX 
|5« 


■'■" 

;^Jh 

l« 

*^i 

M±      ^■mtm- 

.oV 

.f>» 

,0^ 

V 

.0% 

00  X 

fi 

B8 

di 

B8 

cX 

"«,' 

XX 

SX 

S"X 

?7 

«i 

iV 

tiX 

'■'  f  -^ 

Q 

I' 

vJwi. 

vx 

38 

c;i 

SR 

♦I 

ir 

£«  Jl 

iX 

PQ 

ax 

TV 

f,X 

■•' 

■^ 

#b 

* 

3 

OOi 

VI 

■">/'• 

'51 

81 

vv 

5i 

X 

*Q 

S3 

i^i 

da 

i.  1 

i^ 

Y 

a 

i^Q 

ax 

58 

*-! 

.X:   ,,'^ 

Li 

£4^ 

ox 

?.i 

sa 

^i 

XV 

81 

Cc 

XX 

s 

|kfi 

ax 

V* 

8 

3 

as 

a 

d 

*e 

3X 

VY 

Si 

1^ 

SI 

,-'■  -■ 

a 

%' 

?v 

sx 

':.d 

0.. 

5\.~ 

e 

«w 

3 

y 

Sd 

i^j.. 

3a 

11 

63 

01 

x^^ 

T 

T 

rl*9        ax      -5^X8        M      >8d        XX      >€?.•        8  i»?siii§vA 
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Table  19a 

Breaicdown;      Si2«  of  town  5000  -   10,000. 
number  of  raters:      7. 
Evaluation  for  aocuraoy. 


Oorrftcted  Sequence 

"Ratlnj^a        Shown, 


161 

149 
145 
137 
127 
123 
lOG 


102 


99 
SB 
84 
82 


13 
4 

11 
8 
I 
5 

10 

6 

3 
9 


Aver'^p;*: 


454 

£71-^ 

41016 

4304 

Ho. 

^. 

*To. 

<> 

No. 

4^0 

vo. 

4 

2 

39 

2 

39 

3 

39 

B 

B6 

0 

0 

1 

14 

2 

39 

8 

8S 

•3 

43 

Z 

43 

4 

57 

6 

86 

1 

14 

4 

57 

S 

68 

O 

86 

2 

39 

3 

43 

4 

57 

7 

IOC 

4 

57 

5 

71 

5 

71 

6 

86 

S 

29 

4 

57 

5 

71 

6 

86 

3 

39 

3 

43 

4 

57 

8 

86 

3 

29 

3 

29 

5 

71 

6 

B6 

1 

14 

1 

14 

1 

4^ 

43 

6 

86 

1 

14 

'5 

29 

3 

43 

4 

57 

1 

K* 

42 

43 

3 

43 

5 

71 

3 

rM, 

394 

4 

5»<. 

8 

83« 

sa 


iirfff  sfefuT 


;s 


-■?      #      ■  s#. 


0<ji  Vv*  -# 

■9^      '■'8        '■■IT        2  I?        a  T."^ 


mtIM 

't-n    ii'ifP 

iV  J/  ^-^J  !  V  «■>  r.  •)-  ^ 

^■^l^i^^ 

!}to    'X«yu)«'ifl! 

.'^■■■■ 

i'j.C'Op,   X'©'*  r 

:                     .t;'' 

8i 

7f00 

Si 

X8I 

i^ 

6AX 

iX 

v«x 

/i 

TSi 

i' 

,^3 

5S1 

OX 

90X 

■ 

90 

& 

8t> 

L 

e 

>e 

V* 

V 

S8 

8 

:»3ifta«»A 
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Table  19b 

breakdown:   Hize  of  town  5,000  -  10,000. 
number  of  raxera;   7. 
Svaiuattoi;  for  oousiatency. 


■  ■  ■■■    -^e 
■ .  „. j:  b 

rieauence 
•nown 

i5 
No. 

.1 

No. 

.0 

tic 

No.      % 

153 

12 

1 

14 

3 

43 

4 

57 

86 

138 

4 

3 

23 

4 

57 

6 

88 

6 

86 

140 

11 

3 

43 

4 

57 

4 

67 

'3 

86 

137 

8 

1 

14 

4 

57 

8 

83 

S 

86 

124 

I 

3 

43 

4 

57 

5 

71 

7 

100 

123 

5 

4 

57 

5 

71 

5 

71 

6 

86 

117 

10 

3 

43 

3 

43 

3 

43 

6 

86 

108 

8 

3 

43 

43 

4 

57 

6 

P6 

104 

6 

3 

43 

4 

57 

4 

57 

6 

86 

liO 

S 

5 

71 

5 

71 

5 

71 

6 

86 

89 

9 

2 

2IQ 

3 

39 

3 

43 

4 

67 

86 

7 

2 

39 

3 

43 

3 

43 

5 

71 

Average: 

3 

38>. 

4 

53f: 

4 

62f> 

6 

SZ' 

-   OOG.c  xt#o^   \o  «  -  • 

II  0*1 

8  WX 

I  #S;I 

e 
or 

a  ^01 

S  Oil 

e  es 

IV        a  vr        i  .  6  i.S        j;  T  88 


c/S 

w 

\ . 

va 

^ 

U 

£ 

a 

S8 

d 

Vd 

^ 

>I 

I 

•J 

* 

6i^ 

53 

a 

iV 

a 

IV 

a 

va 

> 

ae 

8 

K> 

c 

5^ 

5 

Sl» 

?; 

r^q 

C," 

V5 

^ 

^j^ 

J 

(Si^ 

/ 

as 

e 

V3 

^ 

Vc 

J' 

Sj^ 

s 

a 

IV 

a 

IV 

3 

IV 

a 

r ;. 

r^.' 

O--: 

c 
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Table  20a 

Breakdown:      Size  of  town  10,000  -  35,000. 
Humber  of  raters:      19 
1Cvalu3.tlcn  for  accuracy. 


Corrected 

Ratings 

Sequence 
Shown 

So.  1^ 

Ko.  ^ 

ro.  ^■ 

420< 

ro.  ^ 

153 

12 

8 

42 

11 

58 

12 

63 

19 

100 

143 

4 

7 

37 

9 

47 

12 

63 

19 

100 

136 

11 

9 

47 

13 

63 

16 

84 

19 

100 

130 

8 

8 

43 

11 

5B 

IG 

84 

19 

100 

ISl 

1 

8 

43 

15 

79 

IB 

36 

19 

100 

117 

5 

8 

42 

10 

53 

14 

74 

19 

100 

104 

10 

5 

36 

B 

43 

12 

63 

17 

90 

97 

2 

6 

32 

8 

43 

9 

47 

19 

100 

94 

8 

7 

C7 

8 

43 

8 

43 

16 

84 

92 

3 

,1 

5 

2 

li 

5 

L^8 

16 

84 

80 

9 

3 

16 

3 

16 

5 

33 

13 

63 

78 

7 

1 

5 

5 

26 

7 

37 

15 

79 

AveraK«: 

6 

311: 

9 

45^ 

11 

SP*^. 

17 

9» 

sa 


V  r>/»  iw  50.>i   "l  o  t    rr :.  f  .t  a  i  n 


>i 

-i-i 

^^i 

''^i 

i  ."1  ■■  . ; 

■^  r.-^-Trv?^ 

.  f^"^ 

'-- 

•>. 

D?! 

.-■..'I 

fx 

■„•  - 

.u 

3<J 

ii 

f  ■ 

e 

KX 

sax 

J  _. . 

v».'" 

rx 

yp 

ij 

T-:^ 

r 

,■• 

^4^  r 

w'.-'X 

^8 

SB 

5^X 

Vv 

<■ ' 

'*'-»■" 

d.';x 

oox 

ei 

*9 

61 

n 

'.  X 

8 

8 

0£X 

■%i. 

t!  ■ .' 

.'  '.  JL 

K.'; 

■ix 

;;"-■.  ii 

B 

•■ 

rrr 

.i 

■  - 

"     1 

1  ;_j_ 

>..:t- 

rj 

VIX 

Qia! 

VI 

* 

J- 

'■• 

i' 

^; 

OX 

*^ox 

;  ,  / 

6/ 

t;^ 

€ 

8ii 

a 

e£ 

VP 

^f?       r'l        r^        r  cf.        '1 


^r.-^ 


sx      ^s 

!?V  rli  V,V 

.•>!^0         YX       VPp 


*'t: 


OS 


•tegHlftVA 
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Table  20b 

^Tftskdovm:      31k©  of  tc?m  10,000  -  35,000 

!3 umber  of  raters:      19 

Evai  ua  c  i o n  for  o on  ai  i.^  t  eno  y . 


Sequence 
Shown 

No. 

No.  i» 

Ifo.  f* 

420f. 

146 

13 

2 

11 

12 

63 

13 

89 

19 

100 

132 

4 

10 

53 

17 

90 

18 

95 

19 

100 

133 
128 

11 
8 

10 
6 

53 

33 

13 

10 

84 

£3 

18 
\ 

15 

95 
79 

IS 
19 

100 
100 

133 

1 

9 

47 

14 

74 

18 

95 

19 

100 

115 

5 

8 

43 

10 

53 

IS 

84 

19 

100 

105 

10 

7 

37 

8 

43 

12 

63 

18 

95 

104 

2 

5 

26 

10 

53 

13 

69 

19 

100 

102 

6 

6 

33 

12 

63 

14 

74 

16 

84 

98 

3 

4 

31 

12 

63 

13 

69 

17 

90 

85 

9 

4 

21 

4 

21 

8 

43 

15 

79 

85 

7 

7 

37 

10 

13 

11 

58 

14 

74 

Average: 

7 

34^ 

11 

':>9i 

14 

74-5 

18 

94' 

QV' 


G 

K^;aR 

-  0( 

JQ,.j 

-    'S 

la  fisjs 

■  -a 

'Id    19C     . 

.Yonn: 

d 

ifit   frn.f  _:♦ 'iMrnvS 

k'. 

'  ^ 

^'■ 

..) 

#/ 

\  f 

c 

"^t 

r^  one 

!:mfiS 

15 'J  P,  ^■ 

» 

.01 

': 

« "^j 

.''j'n 

<•;■ 

.0)a 

ei 

QB 

SI 

o8 

ex 

11 

r 

SI 

3*1 

f>t&^ 

81 

oe 

"i 

sa 

r!A  t 
KJX 

!> 

5>SX 

ei 

m 

4 

ea 

i 
OX 

t: 

6&i 

fitftr 

o*yt 

9f 

d^ 

Bi  ■ 

A-  ■" 

r 

^:sx 

\J       :■ 

i-C 

wi 

V.V 

-■ 

c 

au- 

5^ 

<^i. 

t.d 

Si 

J.t> 

e 

ve 

V 

ox 

d0Xl 

001 

ei 

m 

Si 

01 

ae 

a 

£ 

K>i 

' 

■i:^\ 

i>i 

'.-■'J 

c.   X 

b 

b 

v^aX 

Ota? 

VI 

ea 

f.i 

58 

c  r 

X£ 

^ 

€ 

t?e 

or 

51 

t^ 

s 

I " 

i- 

If 

^ 

e 

as 

-■■' 

;-i 

VI 

■\ 

V 

^8 

*»b 

di 

■>'.  **s" 

.**i 

-{'P^r 

J. .- 

<;s?^s 

'^ 

;9^rTsvA 
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Table  3Ia 

Breaicdcwn:      Sia«  of   iown  25,0-0  -  50,000. 
Number  of  Raters;      10. 
kvaluation  lor  accupao/. 


Correctad  Sequenoe 
RatlngB   Shown. 

47 
Ho. 

4 

flO^ 
Ho. 

^ 

ieoi 

^0.  It, 

153 

13 

4 

40 

7 

70 

7 

70 

iO 

100 

141 

4 

6 

60 

7 

70 

8 

80 

9 

90 

135 

11 

5 

50 

6 

60 

9 

90 

9 

90 

130 

8 

3 

30 

7 

70 

8 

80 

9 

90 
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Table  31b 

Brealcdown:   Sire  of   town  35,000  -  bO,000. 
Kuaber  of  rat^re:   10. 
Evaluation  for  conr^ieteEcy. 


Average 
Ratings 

Sequenc  e 
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Breakdown:      31 xe  of   town  50,000  -  100,000. 
Number  of  raters:      7, 
Evaluation  for  aoouracy. 


Oorr acted  Se-mence 
Ratings       Shown 
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11 
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8 
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breakdown:      Size  of   town  50,000  -  100,000 

tuiber  df  ^ters:      7. 
Ivaluation  for  consistency. 


ATerage 
Rat  ings 
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Table  33a 

Breakdown:   Size  of  town  over  100,000. 
number  of  raters:  11. 
Iraluatlon  for  accuracy. 


Corrected 
Ratings 

Seouenoe 
Shown 
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55 
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Table  33b 

Breaicdoiin:      3is«  of  town  over  100,000. 
Kuaber  of  raters:      11. 

firaluation  for  oouaisteucy. 


Average 

Ratings 

Sequeuce 
3tiowii 

£5^ 
No.  i 

No. 

±10* 

No.  i 

±20% 
Ko.  i 

149 

13 

4 

36 

5 

46 

5 

46 

7 

54 

138 

4 

3 

37 

4 

36 

5 

46 

8 

73 

131 

11 

4 

36 

6 

55 

6 

55 

8 

73 

138 

8 

5 

46 

5 

45 

6 

55 

7 

64 

130 

1 

3 
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5 

46 

7 
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107 
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8 

73 

9 
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73 

101 

3 

4 

36 

4 

36 
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Table  34 


Results  of  AnalyBi8-of-Vari*ince  Teste 


Breakdown 

''oOS 

^•clO 

''OS 

Signifi 

cant 

Area 

3.21 

6.64 

3.95 

Yes 

Sxperienoe 

1.11 

1.67 

2.88 

No 

Education                        6.48 

Concept  of 

••standard  performance*   3.05 

7.11 

8.64 

343-Fo05 
4.84PclO 

4.84 

At  lOft 
At   lOlt 

]fo.    of   employees 

1.03 

3.30 

2.44 

No 

Siae  of  city 

3.03 

2.60 

2.38 

Barely 
1016 

at 

''o05  ^^^  ^olO  iJidioate  oooputed  T*  b  at  tiie  4B^  and  £10)( 
levels  of  accuracy. 

Fo5  is  tabulated  values  of  F  at  the  5%   level  of 
Blgnifioance. 
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Fig.  2 
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Fig.  4 
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Fig.  5 
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Fig.  6 
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Fig.  7 


60   70   80   90   100  110  120  130  140  150  160  170  160   X 


f^ 


71 


Fig.  8 
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Fig»  10 
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Fig.  12 
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Fig.  13 
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Fig.  14 
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Fig.  15 
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Fig.    18 
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Fig.  21 
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ns.    22 
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rig.  23 


Job  description 
L«ft  ba&d. 
Transport  empty  to  pine. 
Qrasp  one  pin. 
Tr&neport  loaded  to  scribe. 
Pod  it  ion  and  trarxsport 
loaded  past  scribe. 
Transport  loaded  to  pin 
roleaHe  area. 
Position  end  release  load. 


Right  hand 
Hold  scribe  through 
entire  cycle. 


Job  description  of  the  synthetic  task:   scribe  end  of 
brass  pine,  using  return  chute. 


d8 


6S  .^it 
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,%tUiio  flxu/ei  :^tsu   ,8niq  Bi^id 
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APPEITDIX  B 
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a   XI(Ti?JF<?'?A 
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kathsuatioal  procedure 

Mathematioal  procedures  involved  the  oomputiug  of  the 
true  paoe  or  '^corrected  rating*  for  each  paoe;  the  ubo  of  the 
•analysis-of -variance"  technique,  and  fitting  a  line  to  a 
eeriee  of  points  by  the  tnethod  of  "least  tsquares".   A  short 
discussion  on  these  procedures  follow. 

1.   Computing  the  true  pace  or  •'corrected  rating". 

a.  Designate  "corrected  rating"  as  Xj^. 

b.  Obtain  the  average  fraae  count  per  cycle  for 
•aoh  pace.  De&ignate  the  frame  count  as  Hq. 

c.  To  any  one  of  tLe  paceB  a^aigu  the  value  100. 
Let  this  be  pace  number  10.   Designate  this  as  X|q*100. 

d.  Since  each  pace  bears  a  fixed  relationehip  to 
all  other  paceH,  the  relationship  beinf-  the  ratio  of  their 
fespective  frame  counts,  we  have  i.he  er^ation  X^  x  %=C,  a 
oon^tant.  We  now  find  XJ  for  each  pace  from  this  ea.uation 
and  XJoSlOO. 

e.  The  "corrected  rating**,  Xj^,  bears  the  following 
relationehip,  Xn-ftKA,  where  K  is  a  conttant.   A  iormula 

for  deriving  K  will  be  given  in  a  nubsecuent  paragraph. 

f.  lith  the  Value  of  K  determined,  compute  Xn, 
The«e  values  of  Xj^  are  in  a  direct  ratio  with  each 

other  according  to  their  respective  frame  count  per  cycle, 
are  related  to  the  raters*  average,  and  represent  the  near- 
est approach  possible  to  a  true  rating. 
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9di    vv>    73  1.    Adt    ;eoj(js   uOise  lol   ^gniiTAi  b«;ro9i'xoo*    to  s&Aq  9ui3 

iTioila   A      .^a«i«jup«  tSBSl"   lo  boii^Offl  Bdt  xd  Bfnioq  J.o  aelies 

.noliol:  aeii/baooiq  •sdi  v.idsuoBlI 

.^V  84  ^nijoo  9aai^  ddJ^  9:rAn?i|/a»a  .saisq  no  ■■■ 

.001  «jtfia7  ed^  xisieaji  seojsq  ftii^  lo  sao  vnn  oT  .o 

.GOI^qU  &«  ald^  s^iiii.ilsdC      ,01  leuHiviti  :»o^q  eo   aidi    I0J 

0^  qixlaaolJisXei  bexll  b  e             ^o^q  ilo^e  soaJtB  «b 

iioX;faup«   airfc    not^  isoisq  riojas   10I  ^X  baJt  x  won  ^H      ,taMt^£iot> 

.OOXS^jX    bAjB 

tgniwoIXol  91.it  a  •gnX^Bi  be^osTToo*  sriT     ,8 

Biuatot  A      .;^adJ:Jnoo  s  •I  J  si&dw    .itXSiV-    t^l^dQaoltuLBl. 
.riqi8T5P»Ti»q   ^a9ur)9&du»  b  at  a^vi^  ed  XXfw  1  ^irliBb  ro\ 

d08^  d*l»r  oX^fli  ^oailb  b  fli   stb  j^X   lo  aauXdV  9b 
, 8X0X0  T8q  ^auoo  sounit  8Tfiao8q8»l  it:^-ft   or   vir/hiooc,   lar.'o 
-ia8i]  81I*   ;rn8aeTq9'^   baa    ,83bi»V8   *8i»Jai  ail-'   oj  i;fl;r^-Lo'j:  9X8 

^Bi  8x/T:t  B    o^  oXdXaaoq  iio«o7.qqB  ^88 
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Proof  that  K-z- 


Kz  Tills  proof  was  derived  by 

Dr.  M.  K,  Mundelr'^rof  esoor  of  InduBtrlal  Engineering, 
General  Engineering  Departnent,  Purdue  UniverBity. 


each  paod. 


Problem:   To  find  the  beat  eet  cf  X  valueB  about  y  =  X, 
OiTon:   X,,  X^, ,  X^^,  the  corrected  ratings  for 

0 


Ni 


ll,   Yg, ,    Yj^,    the  average  of  the  raters' 

eatimate  for  each  pace. 

K  is  a  const pjit  to  be  selected  such  that  ?d 
equals  a  ainiaum,   where  d  is  the  variation  about  the  line 
Y;r  X. 

Proof:      Sd  -   (Yx  -  KXp      +      (Yg  -  KXp   4. ..4   (Y^-KX^). 

^d^=  Yi  -   3KXjr^   4  K^X{^t    ...    ♦    (Yj^  -   KX^)^. 
The  value  of  K  which  will  permit  ^d*^  to  be  a  ainiauoi  is 
obtained  by  setting  the  partial  derivative  of  ^d^  with 


ee 


8^ 


fJkM  ^pAX 


AXj*. 


(X)* 


•X  =  Y  tuodB  aeul^v  X  lo  tea   tasd  9il;r  Oi-iil  oT      :.«»ldoT<l 
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«»&jiq  cLon9 


JL 


•=iX 
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^tZ  tadS  douB  besomlsi  Bd   ot  tnMt»aoo  a  at  "% 
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roBpect  to  K  equal  to  0  and  solving  for  K, 

Solving  for  K  *e  have, 
_^X'Yi  +  X4Y3  +  ...  4  X^Yn 

K  ^  -~— nrmn ir-ni — r  -n  .  * 

}[p  +  x|^TT774l^ 

n-.i  ^^n 

§  X»^ 
nil 
2.     Analysi8-of-varlanoe  technique. 

Details  of  the  analyaie-of-variance  teste   are  not   pre- 
sented here       but  It  can  be  shown  that  if   we  have  the  variable 
X  arran(5«d  in  the  following  manner, 


ll 


^3 


•    •   •    •  A 


Ij 


^31     ^23'*-'^3;j 


Hi     *i2 5Cij 


*al     ^a2 *aj 


.1         .2  .J 


lb 
^3b 


Xib 


%. 


ab 


'.b 


and  if   we  assume  theue  values  of  X  are  random  sample 
values  from  a  aorsial  population  with  mean  n  and  variation  C,^ 


10  Paul  G,   Hoel,    XntroQUOtion  to  aifctUematlcal  Sigtiaiice. 
p.   154-161,   John  Wiley  *  Sonu,    Inc.,   Hew  York,   1949. 
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,  a  •.  -i 


5^   V-"    a,-(ot,   ,ldl-*ex    .q 
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a  2(1  ,  -  I)^      (a-1)  (to-1) 
then  F„ :ill — 'J ^ -. ,,^^ 

The  numerator,  with  (t>-*l}  degreei^  of  freedon,  aeaaures 
rarlatlon  between  coluane.   The  denominator^  with  (a'-l)(b-'l) 
degrees  of  freedoa,  neasurea  the  "experimental  error",  that 
ie,  Variation  in  the  fundamental  variacie  sfter  ro»,  and  column 
effects  have  been  eliminated. 

In  thiB  thesis,  a  i»  the  number  of  peces,  b  is  the  number 
of  breakdowns,  X  is  the  percentage  of  raters  within  ^5*^  of  the 
corrected  rating  or  any  designated  level  of  accuracy, 

Fq  Was  computed  et  the  ±^   and  ilO^  levels  of  accuracy 
for  each  category  of  breaJcdown  and  w&a  oomparea  with  the 
tabular  value  of  F  at  the  5^  level  of  significance. 

3.  Fitting  a  curve  to  a  set  of  points  by  the  method  of 
•least  squares*. 

Details  of  the  method  of  "lea^t  squares*  will  ivot  be 
presented  here  but  involveB  xhe  use  of  the  following  equations. 

^(•s    aH  +  aft. 
f  XY  5  a^X  -f  m  JX^. 

a  "intercept,   msslope. 
With  the  Values  of  a  and  m  determined  we  substitute  them 
In  the  general   equation  for  a   straight  line,    Y  a  mX  •♦  ai    and 
obtain  the  line  best  fitting  the  available  points. 


11  Ibid.,   p.    79. 
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(I-<i)(X-ji)  rfifl"   ,ro 3 tiat moaf^t  9cn     .^amnoo  a«©w;red  aoi^tAlxsv 

•rii   to  ^d^:  atditw  «i»*jbi  lo  9i^dia90zsr   -^t  al  X   ^artwc^jUe-xcr  lo  I 
X0/aoD05  to  oXevsX  ^ff-  bfla  i^t  ^At  S^  b©:M/q«oo  aa»  ^t 


ai:j  iiJ  .  xa^ixi'  ■'  ^      -'  xioji*^"'^   ^^'^-'^  tol 

.»o««oXlXaB48   ic   iev^  aiiiT   ^#  *«  ^o  aiJi*iV  XAii/^J 

lo  bod;?9jr  ari^r  xc*  ainioq  to   t^a  «   o^  ^viuo  f  ^nltJlt    .5 

9d  tOii  XX tw   "fidt^upa  ^aB©X"    to  ;  ■'■  ©43    to   aXiaJ-fta 

"^^.afl.^t.^ayns   aa^wcfXot  ed:  to  aau  ©ili   saviovfii   Jocf  ©i^xi  b©J.(»a»xq 

.sqolacw    ,^q80i«^aXs  jb 
a^dt  m^uti&^.iun  9w  b«nrferf-»J*h  «  ?>ii»  b  to  BBular  9dt  dttW 
bam    ,J5  i'   •  .- s  2    ,©iTiX  3'iai^x...ii ?i   i.  lot  notiouiye   XJjiaii©3  ©rf^  fll 
.eliiloq  ©IcfAll  i;r;rit  #a©d  ©niX  9dt  alat^o 


.ev    .q    , .btdl   XX 
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